Automated robotic microscopy systems

ABSTRACT

The present disclosure provides automated robotic microscopy systems that facilitate high throughput and high content analysis of biological samples, such as living cells and/or tissues. In certain aspects, the systems are configured to reduce user intervention relative to existing technologies, and allow for precise return to and re-imaging of the same field (e.g., the same cell) that has been previously imaged. This capability enables experiments and testing of hypotheses that deal with causality over time with greater precision and throughput than conventional microscopy methods.

CROSS-REFERENCE

This application is a continuation of U.S. application Ser. No.14/737,325, filed Jun. 11, 2015, now U.S. Pat. No. 10,474,920, which isa continuation-in-part of International Application No.PCT/US2013/075045, filed Dec. 13, 2013, which application claims thebenefit of U.S. Provisional Application No. 61/737,683, filed Dec. 14,2012, which applications are incorporated herein by reference in theirentirety.

INTRODUCTION

Recent advances in microscopy have contributed to the analysis ofsamples and systems, including biological samples and systems, withimproved efficiency. For example, inverted microscope configurations andcomputer control for automatic focusing and microscope stage positioninghave been developed to facilitate the repeated imaging of biologicalsamples. U.S. Pat. No. 4,000,417 describes a computer automated systemthat permits re-viewing of previously viewed cells on previously usedslides. Further, U.S. Pat. No. 7,139,415 describes a robotic microscopesystem and methods that allow high through-put analysis of biologicalmaterials, particularly living cells, and precise return to andre-imaging of the same field (e.g., the same cell) over time.

SUMMARY

The present disclosure provides automated robotic microscopy systemsthat facilitate high throughput and high content analysis of samples,including biological samples such as living cells and/or tissues. Incertain aspects, the systems are configured to reduce user interventionrelative to existing technologies, and allow for precise return to andre-imaging of the same field (e.g., the same cell) that has beenpreviously imaged. This capability enables experiments and testing ofhypotheses that deal with causality over time with greater precision andthroughput than existing technologies.

For example, embodiments of the subject systems provide a degree ofthroughput and analytic capability not possible with the systemsdescribed in U.S. Pat. Nos. 4,920,053; 5,991,028; 4,705,949; 5,594,235;6,005,964; 5,861,985; 6,049,421; 6,246,785; 4,958,920; and 7,139,415; orthe systems described in the publications: Anal Biochem 2001 Jun. 15;293(2):258-63, Ultramicroscopy 2001, April; 87(3): 155-64, FoliaHistochem Cytobiol 2001; 39(2):75-85, Trends Cell Biol 2001 August;11(8): 329-34, J Microbiol Methods 2000 October; 42(2):129-38, J ImmunolMethods 1999 Nov. 19; 230(1-2):11-8, Environmental Health Perspective1999, November; 107(11), and Nature 2001 May; 411: 107-110; thedisclosures of which are incorporated herein by reference.

In certain aspects, systems of the present disclosure include an imagingdevice including a sample holder; a transport device configured to placea sample in the sample holder; a processor in communication with theimaging device and the transport device; and memory operably coupled tothe processor, wherein the memory includes instructions stored thereonfor acquiring an image of the sample, wherein the instructions, whenexecuted by the processor, cause the processor to: move the sample viathe transport device to the sample holder of the imaging device;identify a fiduciary mark on the sample using the imaging device; movethe sample holder so that the fiduciary mark is in substantially thesame position as in a reference image; and acquire an image of thesample using the imaging device.

In other aspects, systems of the present disclosure include an imagingdevice and a robotic arm configured to automatically retrieve a samplefrom a first surface and place the sample on the imaging device, whereinthe system is configured to automatically identify a fiduciary mark onthe sample, move the sample so that the fiduciary mark is insubstantially the same position as in a reference image, and acquire animage of the sample. The robotic arm may be configured to interact witha plurality of imaging devices (e.g., a system includes two imagingdevices and one robotic arm).

Systems of the present disclosure may be used with a variety of sampletypes. In some aspects, a sample includes biological material, such asliving cells (e.g., neurons) and/or tissues. Biological material may beobtained from an in vitro source (e.g., a suspension of cells fromlaboratory cells grown in culture) or from an in vivo source (e.g., amammalian subject, a human subject, etc.). Samples of interest furtherinclude non-biological samples, such as those from chemical or syntheticsources. Samples may be present on a variety of differentsample-containing means, such as plates, including multi-well plates ofany convenient number, composition, color, and the like.

Systems of the present disclosure may include a bulk sample storagesubsystem. In such systems, a transport device may be configured to movea sample from the bulk sample storage subsystem to the imaging device,such as to a sample holder of the imaging device. Such movement may beautomated, such that the transport device is controlled by a processorprogrammed to control the transport device. The bulk sample storagesubsystem itself may store one or more samples, such as 5 or more,including 20 or more, 40 or more, or 80 or more. The samples may be keptat desired conditions (e.g., a desired temperature, humidity, etc.)within the bulk sample storage subsystem. In certain aspects, suchdesired conditions may be user-defined and/or under closed-loop control.Aspects of embodiments of systems of the present disclosure includesystems in which samples are stored under homogeneous or heterogeneousconditions, e.g., a first population of the samples are stored underfirst desired conditions, a second population of the samples are storedunder second desired conditions, etc. In such embodiments in which thesamples are stored under heterogeneous conditions, the first, second,etc. conditions may differ from one another by one or more properties,such as temperature, humidity, and the like.

Transport devices used in systems of the present disclosure may varygreatly. In certain aspects, a transport device is a robotic arm.Robotic arms of interest may include one or more sample engagementelements, such as grippers, which facilitate the acquisition ortransport of a sample. In certain embodiments, grippers may exert alateral pressure on a sample, and may be adjustable so as to engagesamples of different sizes. Aspects include systems comprising aplurality of transport devices, including 2 or more, such as 2 to 4, 4to 6, 6 to 10, or 10 to 15. In such embodiments, the transport devicesmay be identical (e.g., all transport devices are the same model ofrobotic arm) or different (e.g., different types of transport devices,different models of similar transport devices such as robotic arms,etc.).

In certain aspects, systems of the present disclosure may include one ormore elements for identifying a particular sample, such as a sampleidentification subsystem. Any convenient means for identifying a samplemay be employed, such as a barcode, QR code, and the like. The handling,imaging, or image processing of an identified sample may each betailored for a particular sample. For example, where samples are presenton multi-well plates, the handling, imaging, and/or image processing ofeach well of the multi-well plate may be tailored for that particularwell.

A variety of imaging devices may be employed in systems of the presentdisclosure, such as imaging devices that include an inverted microscopebody. Imaging devices of interest include, but are not limited to,imaging devices in which acquiring an image of a sample includesdeconvolving a multi-wavelength image into its component wavelengthsand/or obtaining 3-dimensional pixel intensities. A number of componentsmay be included in an imaging device, such as a camera (e.g., an EMCCDcamera), light source (e.g., a Xenon light source with light guide),filters, automated focusing components, and the like, as is describedherein. Systems of interest include, but are not limited to, systemscomprising 2 or more heterogeneous and/or homogeneous imaging devices,such as systems containing 2 imaging devices, 3 to 5 imaging devices,and the like. In such embodiments, the imaging devices may be identicalor may differ from at least one other imaging device in at least oneway, such as the objective power, speed of image acquisition, type offilter(s) (e.g., different filter wheels), type of camera(s), source ofillumination(s), and the like.

Imaging devices may be adapted to include a sample holding device of thepresent disclosure. Embodiments of sample holding devices of the presentdisclosure include two first walls of approximately equal lengthpositioned in opposition, the first walls each defining a cutoutportion, an internal beveled edge and an internal bottom lip portion;and two second walls of approximately equal length positioned inopposition, the second walls each defining an internal beveled edge andan internal bottom lip portion; wherein each of the two second walls areshorter in length than each of the two first walls, and wherein the twofirst walls and the two second walls together define a sample receivingarea. In certain aspects, the device is so dimensioned as to receive asample having a standard size, such as a multi-well plate having a 127.5mm×85 mm footprint. Sample holding devices may include an actuator(e.g., a passive or active actuator) configured to secure a sample thatis placed in the sample holding device.

Systems may include hardware and/or software for performing a number ofadditional tasks. For instance, in certain aspects systems include aprocessor programmed to acquire an image of the sample using the imagingdevice; identify a fiduciary mark in the image; compare the image of thefiduciary mark with a reference image; and move the sample so that thefiduciary mark is in substantially the same position as in the referenceimage. Moreover, in certain aspects systems include a processorprogrammed to process one or more images of a sample, such as byorganizing a plurality of images of the sample; stitching two or moreimages of the sample together (e.g., using rigid or flexible stitching);aligning two or more images of the sample; identifying objects within animage of the sample; tracking an object through a temporal series ofimages of the sample; and/or extracting data from objects identifiedwithin an image.

Also provided by the present disclosure are methods of acquiring imagesof a sample, and methods for organizing and/or processing such images.For example, in certain embodiments methods of acquiring an image of asample include moving the sample using a transport device controlled bya processor to a sample holder of an imaging device identifying, withthe processor, a fiduciary mark on the sample; aligning, with theprocessor, the sample holder so that the fiduciary mark is insubstantially the same position as in a reference image; and acquiring,using the imaging device controlled by the processor, an image of thesample. Also provided are methods for processing one or more images,such as by organizing a plurality of images of the sample; stitching twoor more images of the sample together (e.g., using rigid and/or flexiblestitching); aligning two or more images of the sample; identifyingobjects within an image of the sample; tracking an object through atemporal series of images of the sample; and/or extracting data fromobjects identified within an image.

These and other features will be apparent to the ordinarily skilledartisan upon reviewing the present disclosure.

DEFINITIONS

Unless defined otherwise, all technical and scientific terms used hereinhave the same meaning as commonly understood by one of ordinary skill inthe art to which this invention belongs. Still, certain elements aredefined below for the sake of clarity and ease of reference.

In certain aspects, the systems and methods of the invention involveimaging of one or more cells which are provided on a substrate. In thiscontext, “substrate” is meant to describe the material on which thesample(s) for imaging are provided (e.g., where cells are grown). Thesubstrate may comprise a plurality of wells (i.e., at least two), whichcan be provided in an array format. The term “sample” may be used hereinto refer to a substrate of any type (e.g., a multi-well plate) thatincludes a biological material, such as cells and/or tissues, and/or anon-biological material (e.g., a synthetic, chemical, or othermaterial), thereon. Non-biological samples of interest include, but arenot limited to, carbon nanotubes.

A “multi-well plate” is a non-limiting example of a well-containingsubstrate in which multiple discrete regions are provided, whereby thewells are provided in an array. Another manner of providing discreteregions is presented, for example, in Nature vol. 411: 107-110 notedabove where a monolayer of cells is grown over DNA spots, wherebydiscrete image/analysis areas are provided. A further example is in aDNA or protein array. Substrates can comprise any suitable material,such as plastic (e.g., polystyrene, black polystyrene, etc.), glass, andthe like. Plastic is conventionally used for maintenance and/or growthof cells in vitro, and is referred to in the specification as exemplaryof substrate materials without limitation.

By “well” it is meant generally a bounded area of a substrate (e.g.,defined by a substrate), which may be either discrete (e.g., to providefor an isolated sample) or in communication with one or more otherbounded areas (e.g., to provide for fluid communication between one ormore samples in a well). For example, cells grown on the substrate arenormally contained within a well, which can further provide forcontaining culture medium for living cells.

A “multi-well plate”, as noted above, is an example of a substratecomprising wells in an array. Multi-well plates that are useful inconnection with the methods, systems and/or devices of the presentdisclosure can be of any of a variety of standard formats (e.g., plateshaving 2, 4, 6, 24, or 96, wells), but can also be in a non-standardformat (e.g., 3, 5, 7, etc. wells).

By “discrete region” it is meant a spot or grouping of interest that maybe bounded (as in a well) or simply have a definable boundary, separatefrom other adjacent units. Whether presented in an array or otherwise,such discrete regions are advantageously provided in a preset pattern.Oftentimes, the pattern will be regular and repeating, though it neednot be.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention may be best understood from the following detaileddescription when read in conjunction with the accompanying drawings.Included in the drawings are the following figures:

FIG. 1 is a flowchart representing capabilities of certain embodimentsof the present disclosure.

FIG. 2 is a graphical overview of the components of certain systems ofthe present disclosure.

FIG. 3 schematically illustrates an imaging device (e.g., an opticalscanner or microscope) as may be used in systems of the presentdisclosure.

FIG. 4 is a perspective view of the underside of a multi-well plate asmay be used in growing, storing and observing cells with automatedmicroscopes of the present disclosure. The side shown is that whichfaces to the optics of an inverted microscope for inspection, though theplate itself will generally be right side up in use.

FIG. 5 is a perspective view of the plate-contacting portion of agripper arm, as may be used with a robotic arm in systems of the presentdisclosure.

FIG. 6 is a top view of first and second gripper arms of the type shownin FIG. 5, attached to a robotic arm as may be used in systems of thepresent disclosure.

FIG. 7 is a top view of a plate holder as may be used in systems of thepresent disclosure.

FIG. 8 is a bottom view of the plate holder of FIG. 7.

FIG. 9, Panels A-D are side views showing the four sides of the plateholder of FIG. 7.

FIG. 10 is a top perspective view of the plate holder of FIG. 7.

FIG. 11 is a top view of a plate holder having a mounting point for anactuator, as may be used in systems of the present disclosure.

FIG. 12 is a bottom view of the plate holder of FIG. 11.

FIG. 13 is a top perspective view of the plate holder of FIG. 11.

FIG. 14 is a top view of a plate holder with an actuator, as may be usedin systems of the present disclosure.

FIG. 15 is a bottom view of the plate holder and actuator of FIG. 14.

FIG. 16, Panels A-B are side views of the plate holder and actuator ofFIG. 14.

FIG. 17 is a top perspective view of the plate holder and actuator ofFIG. 14.

FIG. 18 is a block flow diagram of a process for control of a system ofthe present disclosure.

FIG. 19 is a flow diagram depicting an image acquisition processaccording to some embodiments of the present disclosure.

FIG. 20, Panels A-B illustrate the image organization of certainembodiments of the present disclosure. Panel A: Images may be given afile name that includes information about the date of acquisition,experiment name, timepoint, hour, fluorescence channel, well, montageindex, and file type. Panel B is an illustration of an organizationscheme for image files.

FIG. 21, Panels A-B are illustrations depicting construction of amontage image in connection with some embodiments of the presentdisclosure. Panel A: Nine images are taken of an individual well of amulti-well plate. For each image, the areas indicated by dashed linescorrespond to areas of the image that are substantially identical tothose areas indicated in dashed lines of the immediately adjacentimages. Panel B: Graphical depiction of a resulting montage image. Thenine images from Panel A are overlayed, as indicated by the dashedlines, to produce one image. Panel C provides a montage image of asingle well from a 96 multi-well plate. Nine individual images wereacquired using a system of the present disclosure and stitched togetherusing a rigid stitching algorithm as described in greater detail herein.Primary cortical neurons were transfected and imaged as described belowfor FIG. 23. The montage image shows a single well at T=0 using the FITC(green) channel.

FIG. 22, Panels A-B provide block flow diagrams of methods for labelingand/or tracking objects (e.g., cells) in an image.

FIG. 23 provides images of one of several primary cortical neurons thatwere transfected with two plasmids: EGFP and a new mitophagy reporterconstruct MitoEOS2. The FITC (green) channel (top row) shows themorphology of the neuron which can be used as a mask for determiningsignal intensity but can also be used for additional image analysisroutines such as analysis of neurites as a readout of neuron health. Thefluorescence of the MitoEOS2 construct can be irreversibly shifted fromgreen to red upon illumination with blue light. The RFP images (bottomrow) show the same neuron shown in the top row of images red-shifted byexposure to a pulse of blue light at the beginning of imaging. The sameneuron was imaged eleven times with the first seven images taken everyfour hours and the last four images separated by twenty four hours. Thetop and bottom rows are images of the same neuron at T1, T2, T3, T4, T5,and T6, wherein T2 is 20 hr after T1, T3 is 24 hours after T1, T4 is 48hr after T1, T5 is 72 hr after T1, and T6 is 96 hours after T1. Thisfigure demonstrates the ability of systems of the present disclosure toenable experiments and testing of hypotheses that deal with causalityover extended time periods, such as 96 hours or more.

FIG. 24 is a graph showing how autophagy induction mitigates neuritedegeneration induced by a disease model of amyotrophic lateral sclerosis(TDP43 M337V). Primary neurons were transfected with GFP as a control orTDP43 M337V. The disease model neurons were treated with fluphenazine(0.1 μM) or vehicle to determine whether autophagy could rescue TDP43M337V mediated loss of neurites. Images were collected every 24 hourswith the robotic microscope and neurites were quantified using automatedanalysis as described herein.

FIG. 25, Panels A-B show the improvement in segmentation when combingthe results from more than one segmentation pipeline. Panel A: a plotshowing false positive and false negative rates from a pipeline of justintensity measurement using various threshold of intensity to segmentthe cells. Decreasing the false positive rate by increasing theintensity threshold results in a substantial increase in the falsenegative rate. Panel B: plot showing false positive and false negativerates using a fixed intensity threshold combined with spatialsegmentation (minimum area threshold) using one of several spatialbandpasses. The use of intensity and spatial segmentation results inimproved specificity without sacrificing sensitivity as shown by thedecrease in false positive rates without an increase in the falsenegative rate.

FIG. 26, Panels A-E show automated detection of neurites. Panel A: adistance map is created, where the pixel intensity is equal to thedistance between two points on the region of interest (ROI). Darkercolors indicate shorter distances. Panel B: distance map showingneurites. The neuron—with neurites—is overlayed in the center of thedistance map. Arrows show the mapping between a neurite and thecorresponding change in the distance map. In the map, the length of theindentation is equal to the length of the neurite segment, and the pixelintensity equals the width of the neurite segment. Panel C: the initialdistance map shown in Panel B includes distances outside of the neuron.In this panel, the distances are restricted to just those distancesinside the ROI. Panel D: thresholding intensity on a distance map isthresholding for neurite width, selected between 0-30 pixels (0-10 μm).Panel E: Branching segments can be linked using the distance map;segment ends with matching x or y coordinates are branching.

FIG. 27 shows an orthogonal projection of a three-dimensional image of aZebrafish embryo. The larger panel represents the XY axes while the twothinner panels on the sides and bottom represent the XZ and YZ planes,which show the three dimensional structure. The height of the stack is150 μm with each image plane representing a 3 μm thick slice.

FIG. 28 shows a representative phase contrast image. The phase contrastimages are acquired using an objective and condenser aperture. Theseoptical elements enhance or attenuate the light passing through thesample based on a phase shift caused by light passing throughinhomogeneous refractive indices. This enhancement and attenuation basedon the phase shift, greatly increases the contrast of the image, makingsmall details more accessible. The implementation of these elements canbe specified by a user.

DETAILED DESCRIPTION

As described above, the present disclosure provides automated roboticmicroscopy systems that facilitate high throughput and high contentanalysis of samples, including biological samples such as living cellsand/or tissues. In certain aspects, the systems are configured to reduceuser intervention relative to existing technologies, and allow forprecise return to and re-imaging of the same field (e.g., the same cell)that has been previously imaged. This capability enables experiments andtesting of hypotheses that deal with causality over time with greaterprecision and throughput than conventional microscopy methods.

Before the present invention is described in greater detail, it is to beunderstood that this invention is not limited to particular embodimentsdescribed, as such may, of course, vary. It is also to be understoodthat the terminology used herein is for the purpose of describingparticular embodiments only, and is not intended to be limiting, sincethe scope of the present invention will be limited only by the appendedclaims.

For example, reference to a “well” or a “multi-well plate” is madethroughout the specification for the purposes of clarity and convenienceonly, and is not meant to be limiting as to the substrate, since aspectsof the present invention encompass imaging of any discrete region asdescribed herein or otherwise. It should also be apparent from thecontext herein, that many aspects of the invention are applicable toimaging or scanning any region—whether discrete or not. Furthermore,while the invention is described primarily in terms of use withbiological samples and living cells, it may, however, be used forimaging of any types of samples, with biological materials being ofparticular interest. For example, the invention can be used in imagingand analysis of a variety of biological materials, such as cells,particularly living cells; the specification refers to “cells”throughout for the purposes of clarity and convenience only, and is notmeant to be limiting. In addition, the invention can be applied toacquisition and analysis of any suitable optical image, of a variety ofdifferent spectral ranges, e.g., any range of color, produced forexample by, reflected light fluorescent emissions, luminescentemissions, chemiluminescent emissions, etc. Reference is made throughoutthe specification to, for example, phase contrast and fluorescentimages; however, the invention is not so limited. The scope of thepresent invention will be established by the appended claims.

As used herein and in the appended claims, the singular forms “a”,“and”, and “the” include plural referents unless the context clearlydictates otherwise. Thus, for example, reference to “an image” includesa plurality of such images, and reference to “the objective” includesreference to one or more objectives and equivalents thereof known tothose skilled in the art, and so forth.

Where a range of values is provided, it is understood that eachintervening value, to the tenth of the unit of the lower limit unlessthe context clearly dictates otherwise, between the upper and lowerlimits of that range is also specifically disclosed. Each smaller rangebetween any stated value or intervening value in a stated range and anyother stated or intervening value in that stated range is encompassedwithin the invention. The upper and lower limits of these smaller rangesmay independently be included or excluded in the range, and each rangewhere either, neither or both limits are included in the smaller rangesis also encompassed within the invention, subject to any specificallyexcluded limit in the stated range. Where the stated range includes oneor both of the limits, ranges excluding either or both of those includedlimits are also included in the invention.

Unless defined otherwise, all technical and scientific terms used hereinhave the same meaning as commonly understood by one of ordinary skill inthe art to which this invention belongs. Although any methods andmaterials similar or equivalent to those described herein can be used inthe practice or testing of the present invention, some potential andexemplary methods and materials may now be described. Any and allpublications mentioned herein are incorporated herein by reference todisclose and describe the methods and/or materials in connection withwhich the publications are cited. It is understood that the presentdisclosure supersedes any disclosure of an incorporated publication tothe extent there is a contradiction.

It is further noted that the claims may be drafted to exclude anyelement. As such, this statement is intended to serve as antecedentbasis for use of such exclusive terminology as “solely”, “only” and thelike in connection with the recitation of claim elements, or the use ofa “negative” limitation.

The publications (including patents) discussed herein are providedsolely for their disclosure prior to the filing date of the presentapplication. Nothing herein is to be construed as an admission that thepresent invention is not entitled to antedate such publication by virtueof prior invention. Further, the dates of publication provided may bedifferent from the actual publication dates which may need to beindependently confirmed. To the extent such publications may set outdefinitions of a term that conflict with the explicit or implicitdefinition of the present disclosure, the definition of the presentdisclosure controls.

As will be apparent to those of skill in the art upon reading thisdisclosure, each of the individual embodiments described and illustratedherein has discrete components and features which may be readilyseparated from or combined with the features of any of the other severalembodiments without departing from the scope or spirit of the presentinvention. Any recited method can be carried out in the order of eventsrecited or in any other order which is logically possible.

Aspects of the present disclosure relate to an automated or roboticmicroscope system and methods that facilitate high through-put analyseson samples, such as living or fixed biological materials such as cellsor tissues. One aspect of the invention allows for precise return to andre-imaging of the same field of living cells that have been imagedearlier.

System hardware is preferably configured to allow imaging of live cellsgrown on tissue culture plastic that can be maintained for long lengthsof time (days to months) in tissue culture dishes. By growing cells on asubstrate (e.g., tissue culture plastic), cell positions becomerelatively fixed with respect to the substrate, which permits subsequentreturn to precisely the same field of cells.

The invention is implemented by way of hardware, optionally as describedbelow, and computer programming. Programming embodying the features ormethodology described herein may be originally loaded into the automatedmicroscope, or the microscope may be preprogrammed to run the same. Suchprogramming, routines and associated hardware constitute various “means”as may be referenced in the claims made hereto. For example, theprogrammed computer referenced herein comprises a means for directingthe action of the various controllers provided. Associated programmingcan be recorded on computer readable media (i.e., any medium that can beread and accessed by a computer). Such media include, but are notlimited to, magnetic storage media, such as floppy discs, hard discstorage medium, and magnetic tape; optical storage media such as CD-ROMsand DVDs; electrical storage media such as RAM, ROM and EPROM; andhybrids of these categories such as magnetic/optical storage media.

Various aspects of the system and methods of the invention will now bedescribed in more detail. Such descriptions are followed by Examplesproviding additional, optional aspects of the invention.

Systems

As described above, aspects of the present disclosure relate to anautomated or robotic microscope system and methods that facilitate highthroughput analyses on samples, such as living or fixed biologicalmaterials such as cells or tissues. In certain aspects, such systems ofthe present disclosure may include hardware and/or software thatfacilitates bulk sample storage, removal and identification of aspecific sample from the bulk sample storage, placement of the sample onan imaging device, aligning the sample on the imaging device, imagingthe sample, and/or returning the sample back to the bulk sample storage(see, e.g., FIG. 1). In certain embodiments, systems further include thecapability of processing the image(s) taken of a sample.

Aspects of embodiments of the systems of the present disclosure includesystems with one or more subsystems with components for carrying out theaforementioned steps. For instance, in certain aspects systems of thepresent disclosure may include one or more of the following: a bulksample storage subsystem; a sample identification subsystem; a sampleplacement subsystem; a sample alignment subsystem; a sample imagingsubsystem; an image processing subsystem; and a sample transportsubsystem, each of which are described herein.

A system, or a subsystem thereof, may be controlled by one or moreprocessors configured to control the system and/or subsystem. In certainaspects, the processor(s) may execute instructions from one or moresoftware modules to cause the system or subsystems thereof to facilitatebulk sample storage, to remove and/or identify a specific sample frombulk sample storage, to place a sample on an imaging device, to align asample on an imaging device, to image a sample, to place a sample intobulk sample storage, and/or to process image(s) taken of a sample. Asoftware module may reside in RAM memory, flash memory, ROM memory,EPROM memory, EEPROM memory, registers, hard disk, a removable disk, aCD-ROM, or any other form of storage medium known in the art. A storagemedium may be coupled to the processor such that the processor can readinformation from, and write information to, the storage medium. In thealternative, the storage medium may be integral to the processor.

In certain aspects, one or more samples (e.g., one or more multi-wellplates containing biological material, such as cells) may be stored in abulk sample storage subsystem. A processor may be configured to causethe bulk sample storage subsystem to heat, cool and/or maintain thesample(s) contained therein at a given temperature(s). A processor maybe configured to cause the bulk sample storage subsystem to remove asample stored therein, such as by using an automated arm containedwithin the bulk sample storage subsystem.

The sample removed from bulk sample storage may be transferred, e.g. toa sample identification subsystem or to an imaging device. In certainaspects, such transfer is achieved using a sample transport subsystem,which may include one or more components (e.g., a belt, robotic arm, andthe like) controlled by a processor for moving the sample.

A sample identification subsystem may contain one or more sensors (e.g.,a barcode reader, optical sensor, and the like) that may detect one ormore distinguishing marks on the sample (e.g., a barcode, QR code, andthe like). The sample identification subsystem may be in electroniccommunication with a processor, configured to receive the informationfrom the sensor(s) so as to identify the particular sample. Once asample has been identified, the processor may apply one or moredifferent parameters (e.g., from a parameter file) to tailor thehandling, imaging and/or image processing for that particular sample.

In certain aspects, a sample may be placed on an imaging device (e.g.,an optical scanner, microscope and the like) by a sample placementsubsystem. The sample placement subsystem may include one or morecomponents (e.g., a belt, robotic arm, and the like) controlled by aprocessor for moving the sample into position on the imaging device. Theparticular components included in a sample placement subsystem may varybased upon, for example, the specific sample type, the particularimaging device, the distance to be moved, and the like. For instance, incertain aspects, a sample placement subsystem may include the use of aplate holder (e.g., a plate holder as depicted in any of FIGS. 7-17),into which a sample (e.g., a multi-well plate containing biologicalmaterial thereon) is placed using a robotic arm. Once placed on animaging device, systems of the present disclosure may include a samplealignment subsystem to refine the alignment of the sample placed on animaging device. Such sample alignment subsystems may include, forexample, components to move the imaging stage, to move the imaging lensor camera, software modules, and other components, as described herein.

Systems of the present disclosure include at least one imaging system.In certain aspects, an imaging system includes an optical scanner ormicroscope as depicted in FIG. 3, which depicts an inverted microscopebody 2, with objectives 4 positioned beneath the stage 6, that is usedto image a sample and to keep the specimen plane a relatively fixeddistance from the objectives. FIG. 4 shows a dish or well plate 8 withindividual wells 10 for samples. Imaging is generally performed throughthe base material 12 of the culture dish or well plate as will bediscussed further below in terms of reducing phototoxicity. The camera14 (e.g., comprising a CCD (charged coupled device)) is shown placeddirectly beneath the microscope body to eliminate the need for an extramirror within the microscope body that could reduce the amount ofemitted light. A fast, high sensitivity CCD camera with a wide dynamicrange may be used for high throughput capability with computer control,to allow resolving and measuring of objects based on intensity, and sothat less illumination of the specimen is required. Programmed computer16 controls automatic switching (via controller 18) between differentfluorescence excitation and emission filter combinations is achieved byinterposing one position filter wheel 10 or filter wheel and shuttercombination 20 between a light source 22, e.g., a Xenon light source,and a fiber optic (liquid light guide) 24 that carries the light to themicroscope (excitation) and another filter wheel 26 between themicroscope body and the camera (emission). Automated filter changes(again, via controller 18) make it possible to resolve and relatedifferent structures or functional processes using multiple fluorescenceindicators. Additional hardware may include a manual input/controldevice 28 such as a “joy-stick”, touch pad, keypad, or the like in orderto manually scan the plate to verify features though eyepiece(s) 30.Though such features are not required of the present invention, theyprovide a convenience to which many users are accustomed. Also,vibration isolating footings 46 to interface with a table 48 or othersupport surface may be advantageously employed.

Additional hardware which may be utilized in connection with systemfocusing is described below. Such hardware may include an incandescentor LED light source 32 moderated by an electronic shutter 34, which isin turn operated by a controller 36. When the shutter is open, light istransmitted from the source via optics 38 to illuminate the field ofview of the objectives. Such lighting is utilized, preferably inconnection with phase contrast optics where a plastic well plate isused, to enable focusing without the use of the xenon light source. Suchan approach using a secondary light source may be desirable in that verylow intensity (substantially) white light is all that is required toachieve focus. It also avoids dependence on light from fluorescentobjects that may become less numerous or even disappear over time. Incontrast, use of a xenon light source and the fluorescence resultingfrom exposure of a sample to the same requires much greater lightintensity that may result in sample phototoxicity if used in connectionwith system focusing. The focus routines discussed below further limitthe potential effects of phototoxicity (even by virtue of exposure tolight source 32) by minimizing time spent under illumination for thepurpose of focusing. In some embodiments, an automated focusing system,such as the Nikon perfect focus system (PFS) may be utilized asdescribed in greater detail below.

Further, systems of the present disclosure may include an imageprocessing subsystem. An image processing subsystem may include one ormore processors that may execute instructions from one or more softwaremodules to, for example, organize the images for a particular sample;stitch two or more images for a particular sample together; alignimages; identify objects (e.g., cells, such as neurons) within an image;track an object (e.g., a cell, such as a neuron) through a temporalseries of images; extract data (e.g., fluorescence data) from objectsidentified within an image; and/or analyze the resulting images.

FIG. 2 presents a further illustration of embodiments of systems 200 ofthe present disclosure. In the embodiment illustrated here, the bulkstorage subsystem includes an incubator 201, which may store two or moresamples at a pre-determined temperature. A sample is removed from theincubator by a robotic arm 202 contained within the incubator 201, whichplaces a sample on a sample identification system that includes a holderwith a barcode reader 203. The barcode reader 203 reads a specificbarcode placed on the plate containing the sample. The barcode reader203 is in communication with a processor, which is configured toidentify the particular sample based at least in part on the particularbarcode.

The processor instructs the sample transport system, which includes arobotic arm 204, to pick up the sample from the holder with a barcodereader 203, and transport it to a sample placement subsystem thatincludes a plate holder 205 (e.g., a plate holder as depicted in FIGS.7-17). Once placed, a sample alignment subsystem which includes anelectronic actuator attached to the plate holder 205, and causes thesample to be aligned within the plate holder 205. Further, the samplealignment subsystem causes the imaging stages of the imaging device 206to move to a particular location, based upon the identification of afiduciary mark on the plate, such as a mark that is that is consistentlyset on or into the plate such as alphanumeric identifiers as element(s)44 seen in FIG. 4.

The sample is then imaged by the sample imaging subsystem, whichincludes an imaging device 206 (e.g., an optical scanner or microscopeas depicted in FIG. 3). Once the sample has been imaged, the robotic arm204 of the sample transport subsystem returns the sample to theincubator 201 of the bulk sample storage subsystem. The robotic arm 204may place the sample back on the holder contained in the sampleidentification subsystem, at which point the robotic arm 202 of theincubator 201 may pick up the sample and return it to storage. Incertain aspects, the barcode reader 203 of the sample identificationsubsystem records that the sample has been imaged and returned to theincubator, by communicating the barcode to the processor, which isconfigured to identify the sample and record such information.

Further, the imaging device 206 is in communication (e.g., wired and/orwireless communication) with an image processor 207. In someembodiments, the image processor 207 includes a processor configured toexecute instructions from software modules to organize the images takenof the sample; stitch the images together; align the images; identifyobjects (e.g., cells, such as neurons) within the images; track anobject (e.g., a cell, such as a neuron) through a temporal series ofimages; extract data (e.g., fluorescence data) from objects identifiedwithin an image; and/or analyze the resulting images.

The preceding describes general features and components of certainembodiments of systems of the present disclosure. Specific features andcomponents of the systems are described in greater detail below.

Sample Storage

As described above, system hardware is preferably configured to allowimaging of samples can be maintained for long lengths of time.Accordingly, in certain embodiments a system includes a bulk samplestorage subsystem that facilitates the storage of samples (e.g.,biological materials, such as live cells, on a substrate) for longlengths of time (e.g., days to months).

The bulk sample storage subsystem may include one or more elements tomaintain the sample(s) at a desired temperature, humidity, O₂concentration, N₂ concentration, CO₂ concentration, and the like. Thebulk sample storage subsystem may thus contain one or more heatingand/or cooling elements, humidifying and/or dehumidifying elements, andthe like. Such desired parameters may be maintained using one or moresensors (e.g., a temperature sensor) in electronic communication with aprocessor in a closed-loop fashion. For example, a processor may beconfigured to cause the bulk sample storage subsystem to heat, cooland/or maintain the sample(s) contained therein at a giventemperature(s) by receiving from a temperature sensor the presenttemperature, and activating a heating and/or cooling element to raise orlower the temperature to a desired value or range. Aspects ofembodiments of systems of the present disclosure include systems inwhich samples are stored under homogeneous or heterogeneous conditions,e.g., a first portion of the samples are stored under first desiredconditions, a second portion of the samples are stored under seconddesired conditions, etc. In such embodiments in which the samples arestored under heterogeneous conditions, the first, second, etc.conditions may differ from one another by one or more properties, suchas temperature, humidity, and the like.

A variety of biological materials may be imaged using systems of thepresent disclosure. In certain aspects, the biological materials arecells. Suitable cells include eukaryotic cells (e.g., mammalian cells)and/or prokaryotic cells (e.g., bacterial cells or archaeal cells).Biological materials may be obtained from an in vitro source (e.g.,laboratory cells grown in culture) or from and in vivo source (e.g., amammalian subject, a human subject, etc.). In some embodiments, thebiological material is obtained from an in vitro source. In vitrosources include, but are not limited to, prokaryotic (e.g., bacterial,archaeal) cell cultures, environmental samples that contain prokaryoticand/or eukaryotic (e.g., mammalian, protest, fungal, etc.) cells,eukaryotic cell cultures (e.g., cultures of established cell lines,cultures of known or purchased cell lines, cultures of immortalized celllines, cultures of primary cells, cultures of laboratory yeast, etc.),tissue cultures, and the like.

In some embodiments, the biological material is obtained from an in vivosource and can include materials obtained from tissues (e.g., cells froma tissue biopsy, cells from a tissue sample, etc.) and/or body fluids(e.g., whole blood, fractionated blood, plasma, serum, saliva, lymphaticfluid, interstitial fluid, etc.). In some cases, cells, fluids, ortissues derived from a subject are cultured, stored, or manipulatedprior to imaging using the subject systems.

Embodiments of the systems and methods of the present disclosure involvethe use of an automated liquid handling workstation (e.g., a MICROLAB®STAR or MICROLAB® NIMBUS liquid handling workstation, such as aMICROLAB® STARlet ML 8 96-prep system, available from Hamilton Robotics,Reno, Nev.) to prepare one or more samples. In such embodiments, thesample(s) may be prepared and moved to the bulk storage subsystem (e.g.,using a transport device, as described herein). Such preparation and/ormovement may be controlled by one or more processors, wherein thepreparation and/or movement are achieved in a semi-automated orautomated manner. An example is provided as Example 6 herein.

In certain embodiments the source of the sample is a “mammal” or“mammalian”, where these terms are used broadly to describe organismswhich are within the class mammalia, including the orders carnivore(e.g., dogs and cats), rodentia (e.g., mice, guinea pigs, and rats), andprimates (e.g., humans, chimpanzees, and monkeys).

Accordingly, a range of sample storage means may be employed by systemsof the present disclosure. In some embodiments, systems of the presentdisclosure include an automated incubator configured to interact withone or more additional components of the transport subsystem describedherein, e.g., a robotic arm (e.g., a KiNEDx KX-300-250 robotic armequipped with a plate gripper including first and second gripper arms,available from Peak Robotics, Colorado Springs, Colo.) as describedherein, and referred to in this context as an external robotic arm. Insome embodiments, the incubator includes a robotic access gate, aninternal robotic arm and an externally located transfer nest or dockinglocation which together allow for the removal and return of a sampleplate from and to a specific location in the incubator. The transfernest includes a sample plate platform which provides an interfacelocation between the incubator and the external robotic arm where theexternal robotic arm can pick up and return a sample plate duringoperation of the system. The transfer nest aligns sample plates receivedfrom the internal robotic arm or the external robotic arm for accuratetransfer positioning, and in some embodiments can correct for a sampleplate misalignment of +/−2 mm. For example, in some embodiments, thetransfer nest includes beveled posts positioned around the plate nestand meant to guide the plate into the correct location for transfer toand from the microscope and incubator. Suitable incubators include thoseavailable from LiCONiC Instruments, Liconic US, Inc., Woburn, Mass.,e.g., the STX44-ICBT 70 deg. C. incubator, equipped with a TransferNest™.

Despite the beveled posts around the plate nest meant to guide the plateinto the correct location for transport to or from the microscope, it isstill possible for misalignment to occur if the area of the plate nestis larger than the area of the plate. If physically altering the area ofthe nest to reduce the variability in plate position is not possible,then a number of mechanical solutions are available. For example, insome aspects the robotic transport device, for instance a robotic arm,is utilized to initially push the plate against one edge or corner andset it down before the actual transport movements to or from the platenest. In other aspects, an independent motor, for instance a servomotor, is utilized which is placed on the underside of the plate nest.This motor is attached to an arm which can rotate up to push the plateagainst one edge or corner and then rotate down so that it can notinterfere with the transport movements.

Sample Identification

As described above, systems of the present disclosure may include asample identification subsystem that may contain one or more sensors(e.g., a barcode reader, optical sensor, and the like) that may detectone or more distinguishing marks on the sample (e.g., a barcode, QRcode, and the like).

For example, in some embodiments, the robotic microscopy system of thepresent disclosure includes a bar code reader configured to read a barcode on a sample plate. In some embodiments, the bar code reader isconfigured to read a barcode positioned along the length of a sampleplate. In other embodiments, the bar code reader is positioned to read abarcode positioned along the width of the sample plate. The bar codereader may be positioned to read a barcode on the sample plate while thesample plate is positioned in the transfer nest or docking locationdescribed above. In such embodiments, the barcode reader may bepositioned on the external surface of the sample storage (e.g., anincubator) or attached to a portion of the transfer nest or dockingstation such that the barcode reader is positioned to read a barcode onthe sample plate, e.g., along the length of the sample plate, while thesample plate is positioned in the transfer nest or docking location. Theuse of sample plates having unique barcodes thereon in combination witha barcode reader as described herein allows for accurate tracking ofspecific sample plates as they are transferred to and from the incubatorand the plate holder during operation of the system. Sample plates maybe provided from the manufacturer with barcodes present, e.g., printed,etched, stamped, provided on a label, etc., or they may be addedsubsequently using any suitable method known in the art, e.g., printing,etching, stamping, application of an adhesive label, etc.

In some embodiments, a means other than a barcode is used foridentifying a sample. Any convenient means of identifying a sample maybe used in systems of the present disclosure, such as QR codes, fiducialmarkers, geometric markers, and the like. Such markers may be detectedusing optical sensors and other identification means known in the art.

Transport

As discussed above, in some embodiments a system may include a subsystemfor transporting a sample, e.g., from an incubator to an imaging device.

In some embodiments, the robotic microscopy system of the presentdisclosure includes a robotic arm such as a KiNEDx KX-300-250 roboticarm, available from Peak Robotics, Colorado Springs, Colo., equippedwith a plate gripper including first and second gripper arms. Therobotic arm acts as an intermediary between the incubator and itsassociated transfer nest or docking location (described above) and theplate holder and associated microscope stage (described below). Therobotic arm may include an electric or pneumatic plate gripper, whichmay be configured to engage a plate in either landscape orientation (twogripper arms positioned to contact the sample plate along the shortersides of the sample plate) or portrait orientation (two gripper armspositioned to contact the sample plate along the longer sides of thesample plate) for a generally rectangular sample plate. The robotic armis capable of moving in three dimensions and accordingly manypossibilities exist for the relative positions of the plate holder, theincubator and the robotic arm provided that the maximum reach of therobotic arm is not exceeded.

In certain aspects, a robotic arm may include a plate gripper of thetype depicted in FIGS. 5 and 6. The plate gripper may be sized andshaped to facilitate the deposition and/or retrieval of a sample plateto and/or from a transfer nest or docking station of a sample storagedevice and a plate holder as described herein. Turning to FIG. 5, agripper arm 300 may include a plurality of extensions 304 (e.g., screws)that contact the periphery of a plate. The extensions 304 may preferablybe adjustable for depth, such that when closed the gripper arm contactsthe plate via the extensions on a vertical surface of the plate, withoutcausing damage to the plate. In certain aspects, the extensions 304contain a surface treatment, such as an adhesive, etching, high-frictionmaterial, and the like, so as to facilitate securely holding a plate. Incertain aspects, the extensions 304 may be manually adjustable. Forexample, in certain embodiments the extensions 304 include screw sidesthat may facilitate the lengthening or shortening of the extensions byturning the screws. In other aspects, the extensions 304 may beself-adjustable. By self-adjustable it is meant that the extensions mayadjust to accommodate plates of varying sizes without additionalintervention by a user. For instance, in certain embodiments theextensions are mounted using springs configured to allow the extensionsto move horizontally to accommodate plates of slightly varyingdimensions.

FIG. 6 depicts a top view of first and second plate grippers 301. Aplate may be retrieved by the gripper arm by being placed between theplate grippers 301, which move towards each other so as to exert alateral pressure on the plate so as to secure it for transport. A platemay be deposited by the gripper arm by moving the plate to a desiredlocation and causing the plate grippers to move away from each otherthereby releasing the lateral pressure on the plate so as to deposit itat a desired location. The grippers may be configured to engage a platein either landscape orientation or portrait orientation for a generallyrectangular sample plate.

Imaging System

Systems of the present disclosure may include an imaging system. Incertain embodiments, an imaging system includes an optical scanner ormicroscope as depicted in FIG. 3.

As depicted in FIG. 3, imaging systems of interest may include aninverted microscope body 2, such as an Eclipse Ti-E/B inverted researchmicroscope, available from Nikon Corporation. Such a body can be used totake advantage of the extra long working distance lenses provided bylonger tube length for the objective 6. This makes it possible tocapture a focal plane that is farther (many millimeters) away from thetip of the objective but still have a relatively good numericalaperture. The relatively long working distances offered by the setupallows focusing beyond the floor of the tissue culture plate 8, intothick specimens such as transfected neurons within a brain slice withoutbumping the objective into the dish. Generally, such samples range inthickness from about 50 to about 400 microns.

In certain aspects, the objective 6 is selected so as to balance a needfor acquiring a large field of view and high resolution images. Incertain aspects, the objective 6 is a Nikon Fluor Series microscopeobjective, such as a 20× Nikon CFI Plan Fluor ELWD objective, with a0.45 numerical aperture (NA) and a 6.9-8.2 mm working distance.

In other aspects, a 4× objective may be used. A 4× image is especiallywell suited for counting cells (e.g., measuring survival) and for somemeasurements of overall morphology. 4× objectives of interest includecommercially available objectives such as those from Nikon and Olympus,such as an Olympus Uplan SL 4×\0.13\PhL, which transmits roughly thesame amount of light as the Nikon objective. However, the Olympusobjective transmits light more evenly across the field, and thedifference in transmission from the edge of the field to the center ofthe field is twice as good for the Olympus objective as for the Nikonobjective.

Embodiments include the use of relatively high NA objectives. Normally,a relatively high numerical aperture objective is preferred to allowcollection of more light (i.e., form an image with less signal) withbetter spatial resolution. In the microscope system of the invention,however, a relatively low numerical aperture lens can still collectenough light to form an image while providing substantially greaterdepth-of-field such that that the image remains in visible focus over awider range of actual Z-positions, even where the system does notinclude a Nikon TiE Perfect Focus System (PFS; described below). Insystems lacking such a PFS, this allows focusing once per well(preferably, in the center) followed by capture of a series (e.g., 3×3or 4×4) of adjacent fields within the same well, that remain in focus;focusing only once per well cuts the time required to image or scan a24-well plate by one half. This problem is largely obviated in systemsthat include a PFS because the PFS is engaged in a well which maintainsfocus the entire time while imaging the well, and it is only turned offprior to moving to the next well.

Plate Holder

As shown in FIG. 3, an imaging system may include a stage 6 that is usedto image a sample and to keep the specimen plane a relatively fixeddistance from the objectives 4. In some embodiments, the roboticmicroscopy system of the present disclosure includes a plate holder onthe stage 6 configured to interact with one or more additionalcomponents of the robotic microscopy system described herein in anautomated manner. Specifically, the present disclosure provides a plateholder configured to physically engage, e.g., mechanically attach to, amicroscope stage (e.g., an MS2500 XY Flat-Top Extended Travel Stage,available from Applied Scientific Instrumentation (ASI), Eugene Oreg.).The plate holder may be physically attached to the microscope stageusing any suitable attachment means known in the art, e.g., screws,bolts, rivets, interlocking mechanisms and the like.

The plate holder is further configured to facilitate the deposition andremoval of a sample plate (e.g., a multi-well plate as described herein)using a robotic arm (e.g., a KiNEDx KX-300-250 robotic arm, availablefrom Peak Robotics, Colorado Springs, Colo.) equipped with a plategripper including first and second gripper arms, e.g., an electric orpneumatic plate gripper, (e.g., an electric Side-Grip-Servo (SGS) plategripper configured in portrait configuration, available from PeakRobotics, Colorado Springs, Colo.).

In some embodiments, the plate holder includes four walls connected inan approximately rectangular configuration so as to define an internalsample plate receiving area, an internal opening and external, top, sideand bottom surfaces. FIGS. 7-10 show various views of one embodiment ofa plate holder 400 as may be used in systems of the present disclosure.As shown the top view shown in FIG. 7, the four walls of the plateholder 400 include two first walls 415 of approximately equal lengthpositioned in opposition and two second walls 420 of approximately equallength positioned in opposition wherein each of the two second walls 420are shorter in length than each of the two first walls 415. The twofirst walls 415 each include a cutout portion 416, an internal bevelededge 417 and an internal bottom lip portion 418, as best seen in the topperspective view in FIG. 10. The two second walls 420 each include aninternal beveled edge 421 and an internal bottom lip portion 422. Theseelements are provided such that the internal sample plate receiving areais defined in part by a bottom lip portion, which bottom lip portion ismade up of the bottom lip portions of the first and second walls. Thebottom lip portion of the internal sample plate receiving area incombination with the cutout portions of the walls defines the perimeterof the internal opening.

As discussed above, the cutout portions 416 may be positioned in the twofirst walls 415, e.g., to provide for engagement with a plate gripperhaving first and second gripper arms configured to hold a sample platein portrait configuration. However, it should be noted that the plateholder may additionally, or alternatively, be configured to provide forengagement with a plate gripper having first and second gripper armsconfigured to hold a sample plate in landscape configuration, in whichcase the cutout portions would be positioned in the two second walls 420or in both the two first walls 415 and the two second walls 420.

In some embodiments, the cutout portions 416 of the two first walls 415are positioned in an opposed configuration as shown in FIG. 7. However,it should be noted that these cutouts 416 may be modified orrepositioned to address changes in the configuration of the plategripper 400 or plate gripper arms or vice versa. Generally, the cutoutportions 416 are sized and shaped to accommodate the dimensions of thegripper arms or vice versa such that the gripper arms can descendvertically into the cutouts such that the sample plate is positioned onthe bottom lip portion of the internal sample plate receiving area,release the sample plate and ascend vertically leaving the sample platepositioned in the plate holder for subsequent imaging. The cutoutportions 416 are also sized and shaped to accommodate the dimensions ofthe gripper arms or vice versa to facilitate the removal of a sampleplate from the plate holder, in which case the gripper arms descendvertically into the cutouts such that they are positioned on opposingsides of the sample plate. The gripper arms then move towards each otheruntil they contact and grip the sample plate. The gripper arms thenascend vertically thereby removing the sample plate from the plateholder.

As discussed above, the internal sample plate receiving area is definedin part by an internal beveled edge. The internal beveled edge of theinternal sample plate receiving area is generally continuous with theexception of the cutout portions (discussed above) and the optionalarcuate or rounded corners 430 (discussed below). The angle of thebeveled edge relative to the plane of the bottom lip portion of theplate holder is less than 90 deg., e.g., less than 85 deg., less than 80deg., less than 75 deg., less than 70 deg., less than 65 deg., less than60 deg., less than 55 deg., less than 50 deg., less than 45 deg., lessthan 40 deg., less than 35 deg., less than 30 deg., less than 25 deg.,or less than 20 deg. In some embodiments, the angle of the beveled edgerelative to the plane of the bottom lip portion of the plate holder isfrom about 90 deg. to about 20 deg., e.g., from about 85 deg. to about25 deg., from about 80 deg. to about 30 deg., from about 75 deg. toabout 35 deg., from about 70 deg. to about 40 deg., from about 65 deg.to about 45 deg., from about 60 deg. to about 50 deg. or about 55 deg.

In some embodiments the beveled edge of the internal sample platereceiving area extends from the plane of the bottom lip portion of thesample plate receiving area to the plane of the top surface of the plateholder, which planes are generally parallel. In other embodiments, thetwo first walls 415 and the two second walls 420 each include a firstportion which contacts the bottom lip portion of the plate holder at anapproximately 90 deg. angle and a second beveled edge portion thatcontacts the top surface of the plate holder and has a beveled edgeangle as discussed above relative to the plane of the bottom lip portionof the plate holder. The first and second portions of the respectivewalls may meet at a point in the respective walls which is approximatelyequidistant between the planes of the bottom lip portion of the plateholder and the top surface of the plate holder. Alternatively, thismeeting point may be positioned closer to the bottom lip portion of theplate holder or the top surface of the plate holder.

As discussed above, the internal sample plate receiving area is definedby a bottom lip portion, which, in some embodiments, meets a portion ofeach plate holder wall at an approximately 90 deg. angle. The bottom lipportion extends inward, e.g., along its entire length, from theintersection of the bottom lip and the portion of each plate holder wallwhich meets the bottom lip portion at an approximately 90 deg. angle (orother suitable angle as discussed above). The bottom lip portion extendsa sufficient distance to provide a stable base for a deposited sampleplate. In some embodiments, the bottom lip portion extends about 2 mm toabout 20 mm, e.g., from about 2 mm to about 18 mm, from about 2 mm toabout 16 mm, from 2 mm to about 14 mm, from about 2 mm to about 12 mm,from about 2 mm to about 10 mm, from about 2 mm to about 8 mm, fromabout 2 mm to about 6 mm, or from about 2 mm to about 4 mm from theintersection of the bottom lip and the portion of each plate holder wallwhich meets the bottom lip portion at an approximately 90 deg. angle (orother suitable angle as discussed above). As discussed above, the bottomlip portion is interrupted along a portion of its length by the cutoutswhich provide for access by the gripper arms of the plate gripper. Insome embodiments the bottom lip portion has a thickness of from about0.4 to about 0.8 mm, e.g., about 0.5 to about 0.7 mm, or about 0.6 mm.

In some embodiments, in addition to the above components, the plateholder is configured to include arcuate or rounded corners 430 at theinternal junction of the two first walls 415 and the two second walls420. These optional rounded corners 430 may facilitate platepositioning, e.g., during deposition and removal of a sample plate.

The plate holder according to the present disclosure may be made fromany suitable material, e.g., aluminum or titanium, and may be producedusing any of a variety of suitable methods known in the art, e.g.,milling or injection molding.

In some embodiments, the plate holder has an overall thickness ofapproximately 8 mm and is sized and shaped to receive a sample platehaving a Society for Biomolecular Sciences (SBS) Standard 127.5 mm×85 mmfootprint. Such embodiments may be configured to provide an internalsample plate receiving area which includes dimensions defined by theinternal termination of the beveled edge which dimensions are a lengthof approximately 128 mm and a width of approximately 86.1 mm.

The beveled edge of the plate holder provides advantages in that itallows for accurate placement of a sample plate despite slightinaccuracies in the x, y, and/or z plate positioning dimensions whichmay result when using a robotic arm equipped with a plate gripper asdescribed herein to deposit a sample plate in the plate holder. Thebeveled edge serves to “guide” the sample plate into the correctposition in the plate holder for subsequent viewing, imaging and/oranalysis.

In some embodiments a plate holder as described herein may be adapted toinclude an actuator including an arm, which, when actuated, functions topush a sample plate positioned in the internal sample plate receivingarea into a consistent corner of the internal sample plate receivingarea each time a sample plate is positioned in the internal sample platereceiving area. This allows the plate position to be corrected forrotation once positioned in the internal sample plate receiving area. Incertain aspects, the actuator may be passive, such as a passive lever.In other aspects, the actuator may be active, such as an electronicallycontrolled actuator controlled by a processor.

FIG. 11 is a top view of a plate holder 401 that has been adapted forinclusion of such an actuator. FIGS. 12-13 provide alternative views ofthe plate holder 401 depicted in FIG. 11. In the embodiment presented inFIG. 11, the plate holder 401 has been adapted from the embodimentdepicted in FIG. 6 by the removal of a portion of material, into whichan actuator may be placed, shown as the portion 450. The plate holder401 also contains mounting holes 451 for securing such an actuator, andan actuator arm cutout 440 to accommodate an actuator arm.

For example, FIG. 14 is a top view of a plate holder 402 with anelectronically controlled actuator 460 installed. The electronicactuator 460 includes an arm 461 and a control unit, and the plateholder is provided with a space (440), e.g., a carved out space, for theplacement of the arm 461. This allows the arm 461 to be completelyretracted out of the way as the sample plate is deposited and/or removedby the robotic arm. These components are best seen in FIG. 17, which isa top perspective view of the plate holder 402 and actuator 460 of FIG.14. The control unit includes a motor which, when so instructed by aprocessor, causes the arm 461 to rotate in a clockwise direction. Thearm 461 thus pushes a sample plate positioned in the internal sampleplate receiving area into a consistent corner of the internal sampleplate receiving area each time a sample plate is positioned in theinternal sample plate receiving area, thus securing the sample plateand/or reducing the degrees of freedom of the sample plate. Additionalviews of the plate holder 402 and actuator 461 are shown in FIGS. 18-19.

Plate Alignment Algorithm

The programming of the present invention allows imaging a biologicalsample (e.g., cells) and then subsequently returning to and re-imagingthat same biological sample at any time interval. Such activity enablesstudy of cause-and-effect relationships in living cells over days orweeks by returning to image the same cells. The invention may use one ormore reference marks on a multi-well plate to quickly position the platein the plate holder on the microscope stage each time the plate isreturned to the microscope for imaging. The mark may be one that isconsistently set on or into the well plate such as alphanumericidentifiers as element(s) 44 seen in FIG. 4. Alternatively, one or morecustom-applied reference points, marks or structures may be employed.

The mark serves as an internal reference for cells on the plate,independent of the position of the plate within the holder. In certainaspects, once a plate is inserted in the holder by the transportsubsystem (e.g., as described above), the stage is moved without userintervention to locate a mark in the exact same position as is shown ina previously captured reference image. For example, the stage may bemoved without user intervention to the approximate location of the markby means of computer memory and associated control algorithms. Further,by means of computer memory and associated control algorithms, areference or fiduciary point for the substrate employed may be located.An imaging step is performed to locate the reference or fiduciary pointfor the substrate. In certain aspects, a software module is executed bya processor, which directs the imaging device to move the loaded plateto a pre-defined position in X-, Y-, and/or Z-space. This location maycorrespond to an approximate location of a fiduciary mark, used tolocate the plate precisely in space. The location may be refined using asoftware module executed by a processor, which directs the imagingdevice to refine the location of the fiduciary mark. Such embodimentsmay include the use of a scale-invariant feature transform (SIFT)algorithm. In certain instances, a plate alignment algorithm mayincorporate an algorithm as described in Lowe D (2004), “DistinctiveImage Features from Scale-Invariant Keypoints”, International Journal ofComputer Vision 60 (2): 91-110; and U.S. Pat. No. 6,711,293; thedisclosures of which are incorporated herein by reference.

In other aspects, a user may also, or instead, manually move the stage(e.g., with a joystick control 28) to get the mark in the exact sameposition as is shown in a previously captured reference image, and theacquisition programming is started with the mark in exactly the sameposition as the reference image (and therefore the same position eachtime the cells are re-imaged) so that each image in each well is also inthe corresponding position.

Returning to a reference mark and finding a position in relation to thatmark provides one manner of returning to the same field to observestationary or substantially stationary cells at separate time intervals.Where more accurate return to a field is desired or required, furtherrefinement of the process is in order. Systems of the present disclosureare able to align images within at least several pixels, even withaccuracy to a single pixel or in perfect registration utilizingsupplemental mathematical techniques.

In certain aspects, image data obtained may be digitally stored, andconverted to a matrix of values. Signals below a threshold value aretreated as zeros while others are treated as numerical values (e.g.,ones for the sake of simplicity, in which case the matrix will have beenbinarized). Second or subsequent imaging of approximately the sameregion (preferably, as generally identified by use of the referencemark(s) as described above) receives the same treatment. Sincemisalignment of images (as in survivability studies, etc.) to besuperimposed via computer software for analysis, results in zerosmultiplied against numerical values and greater mismatch of the matricesexacerbates this effect, lower sums for the multiplied matricesrepresent less aligned positions. Conversely, a peak or spike representsa maximum value of the sum of matrix numbers—indicating full (or atleast optimized) alignment.

In certain aspects, second or subsequent imaging of approximately thesame region is aligned to prior image(s) using phase correlation toestimate the relative translative offset between the images. Anyconvenient phase correlation approaches may be employed, including butnot limited to those described in Stone H S, IEEE Transactions onGeoscience and Remote Sensing, V. 39, No. 10, October 2001, pp.2235-2242; De Castro E, et al., IEEE Transactions on pattern analysisand machine intelligence, September 1987; and Reddy B S, et al., IEEETransactions on Image Processing 5, no. 8 (1996): 1266-1271; thedisclosures of which are incorporated herein by reference.

It may in some instances be preferred to utilize a subset (e.g., thecentral 80%) of the matrices in the registration process. Such anapproach helps avoid situations where a portion of one matrix is notrepresented in the other matrix (and therefore would not contribute tothe sum to identify a local maximum—unless the matrices were alreadyidentical/aligned) and the potential for unpredictability associatedwith the same. This may be useful in embodiments that do not incorporateSIFT (described above), which is able to align even with partialoverlapping. Furthermore, taking a subset of the available matrix valueslowers computational requirements.

Note that even smaller matrices than the exemplary 80% approach may beemployed—at least to roughly align images. By further reducing thecomputation demand on the system (by utilizing smaller matrix subsets),it becomes increasingly feasible to attempt registration of largersampled areas. Also, with reduced computational demands, it will in somecases be possible to register images that are coarsely aligned (e.g.,initially aligned without involving the reference mark approach).

Such action is followed by an imaging step. Preferably, a phase contrastimage is produced. However, fluoroscopic imaging of the cells may beemployed. In any case, the image is ultimately superimposed with aprevious image that is likewise imaged in relation to the referencepoint. The superposition is preferably accomplished using a matrixregistration technique as described above in which the highest sum ofthe product of two matrices (or matrix subsets) representative of thepixel values is sought. In instances where the information utilized forregistration is a phase contrast image, it should be paired forregistration with another such image previously provided. The use ofphase contrast images over fluorescent-based images is preferred becauselittle, if any, difference should be presented by the phase contrastimages.

When registration is performed in connection with one image andsubsequent imaging follows, these latter-produced images will be alignedor superimposed as well. While it may be preferred to conduct subsequentscans/imaging in such a manner and the computer processor directs stagemovement (generally X and Y-axis movement) to align the images that nopost computer processing is required to align them, an alternateapproach is to perform registration of the images after all imaging iscomplete (i.e., off-line). That is to say, stored image data can bealigned using the matrix approach described. Usually, use of a referencepoint or stored reference position will still be desired for roughalignment, to be followed with fine alignment performed with the matrixmethod. Accordingly, both on-line and off-line registration techniquesare taught hereby.

In an exemplary implementation of this aspect of the invention, computerprogramming directs taking two pairs of images. It directs taking afirst phase contrast image and first fluorescence image, then directsmovement of the system's fluorescence emission filter wheel, followed bytaking a second pair of phase and fluorescence images. Because movementof the filter wheel may be the cause of image misalignment in thereferenced system, each pair of phase and fluorescence images (collectedwhile the wheel is stationary) are aligned. However, the first andsecond image pairs may be misaligned/misregistered with respect to theother, e.g., due to perturbation of the system caused by movement of thefilter wheel. The misalignment, when present, may be correctedautomatically via computer control employing the matrix methodologydescribed above utilizing matrices derived from the more comparablephase contrast images that correspond to the fluorescent images—at leastin terms of their registration. In other embodiments, movement of thefilter wheel does not cause perturbation of the system and correctivealignment as described above is unnecessary.

Camera

Returning to FIG. 3, an imaging system may include a camera 14. In someembodiments, the camera is preferably placed directly beneath themicroscope body to eliminate the need for an extra mirror within themicroscope body that could reduce the amount of emitted light.

In certain aspects, the camera is a CCD camera. Aspects of embodimentsmay include an Electron Multiplying CCD (EMCCD) camera, such as an AndorEMCCD iXon3 888 CCD camera. Cameras of interest further include, but arenot limited, to, Andor EMCCD iXon3 860, iXon3 897, and iXon3 Ultra 897Ultra CCD cameras. An EMCCD can enable a large field of view and highquantum efficiency of light to electron conversion across a broad partof the spectrum. The camera can also be a CMOS camera such as an AndorNeo 5.5 or Zyla 5.5 A CMOS camera enables collection of a large field ofview while imaging at high frequency (100 fps) to study more rapidbiological processes.

Cameras for use in imaging systems of the present disclosure preferablyhave an exposure of 100 ms or less when reading out the full sensor,such as 50 ms or less, including 30 ms or less, 20 ms or less, 10 ms orless, or 5 ms or less. In some embodiments, cameras for use in imagingsystems of the present disclosure preferably have an exposure of 100 msto 5 ms, e.g., 50 ms to 5 ms, 30 ms to 5 ms, 20 ms to 5 ms, or 10 ms to5 ms.

In certain aspects, cameras for use in imaging systems of the presentdisclosure contain 1M active pixels or more, such as 1.5M or more, e.g.,2M or more, 2.5M or more, or 3M or more. In certain aspects, a pixelcorresponds to an actual physical dimension of about 0.3 μm. Further, incertain aspects a camera used in imaging systems of the presentdisclosure include a sensor area of 150 mm² or more, such as about 150mm² to about 175 mm², about 175 mm² to about 200 mm², 200 mm² to about225 mm², about 225 mm² to about 250 mm², about 250 mm² to about 300 mm²,about 300 mm² to about 400 mm², about 400 mm² to about 500 mm², about500 mm² to about 750 mm², about 750 mm² to about 1000 mm², or about 1000mm² or more.

In certain aspects, acquiring an image of the sample comprisesdeconvolving a multi-wavelength image into its component wavelengths.Any convenient means of performing such deconvolving may be employed,with approaches of interest including, but not limited to, thosedescribed in T Zimmerman, et al. FEBS Letters 546: 87-92 (2003); M EDickinson, et al. Journal of Biomedical Optics 8: 329-338 (2003); YHiraoka, et al. Cell Structure and Function 27: 367-374 (2002); TZimmerman. Advances in Biochemical Engineering/Biotechnology 95: 245-265(2005); J M Lerner and R M Zucker. Cytometry A 62: 8-34 (2004); RLansford, et al. Journal of Biomedical Optics 6: 311-318 (2001); R ANeher and E Neher. Journal of Microscopy 213: 46-62 (2003); Y Garini, etal. Cytometry A 69: 735-747 (2006); R M Levnson and J R Mansfield.Cytometry A 69: 748-758 (2006); R H Berg. Journal of Microscopy 214:174-181 (2004); C. Spriet, et al. Microscopy Research and Technique 70:85-94 (2007); C. Thaler, et al. Biophysical Journal 89: 2736-2749(2005); D. Megias, et al. Microscopy Research and Technique 72: 1-11(2009); and Y. Chen, et al. Journal of Microscopy 228: 139-152 (2007);the disclosures of which are incorporated herein by reference. Forexample, in certain aspects, such deconvolving is implemented by linescanning method that uses a vertical illumination slit and grating todisperse the emitted light along the horizontal dimension of the CCD. Inthis case, the CCD is read out as a full image for each slit locationwith spatial resolution along the vertical dimension and spectralresolution along the horizontal dimension. An alternative implementationis having multiple CCDs to simultaneously capture emitted light. In thiscase, a series of filters is used to direct emitted light of certainwavelengths to the different CCDs. With either implementation, linearunmixing can be used to further deconvolve overlapping spectra if thespectrum of each probe in the sample is known.

Light Source

Systems of the present disclosure may include a light source having abroad spectral output. In certain aspects, the light source includes aXenon light source 22 and a fiber optic 24. Light sources of interestfurther include, but are not limited to, LED/solid state light sourcesand lasers.

In certain aspects, the output of the light source is adjustable.Aspects of systems include automated output adjustment. In certainembodiments, a control processor is configured to adjust the output ofthe Xenon light source 22. In certain aspects, the output is tailoredfor a particular sample. For example, a processor may be configured toidentify a sample (e.g., using the sample identification subsystem),which includes one or more parameter files to be read (see, e.g., FIG.18). One such parameter that may be read by the processor may includethe output level of the light to be applied when imaging the sample. Theprocessor may be in communication with the Xenon light source 22, eitherdirectly or via imaging device software, to provide such control.

In certain aspects, a light source may include a filter wheel. Where alight source contains a filter wheel, each filter position may beassociated with a specified intensity level. The specified intensitylevel may be automatically selected when a filter is called.

Embodiments of systems of the present disclosure may include Xenon lightsources having an output range of about 200 nm to about 800 nm,including about 300 to 700 nm.

In certain aspects, the Xenon light source may be Lambda XL lightsource, such as a Lambda XL light source with an integrated 10-Bcontroller for filter wheel and Smartshutter (Sutter Instrument Co.).

Optical Filters

An imaging system may include one or more filters. In certain aspects,the filters may be present in a filter wheel 10 or filter wheel andshutter combination 20. The filter wheels 10 or filter wheel and shuttercombination 20 may be placed between the light source 22 and the camera14.

Filters that may be used in aspects of the present system includesingle-band bandpass filters, edge filters, bandpass clean-up filters,notch filters, dichroic beamsplitters, polarizing filters, and the like.

Filters may be selected to correspond to one or more fluorophores thatmay be present in a sample. Examples of such fluorophores include, butare not limited to, indocarbocyanine (C3), indodicarbocyanine (C5), Cy3,Cy3.5, Cy5, Cy5.5, Cy7, Texas Red, Pacific Blue, Oregon Green 488, Alexafluor-355, Alexa Fluor 488, Alexa Fluor 532, Alexa Fluor 546, AlexaFluor-555, Alexa Fluor 568, Alexa Fluor 594, Alexa Fluor 647, AlexaFluor 660, Alexa Fluor 680, Alexa Fluor 700, JOE, Lissamine, RhodamineGreen, BODIPY, fluorescein isothiocyanate (FITC), carboxy-fluorescein(FAM), phycoerythrin, rhodamine, dichlororhodamine (dRhodamine), carboxytetramethylrhodamine (TAMRA), carboxy-X-rhodamine (ROX), LIZ, VIC, NED,PET, SYBR, PicoGreen, RiboGreen, and the like.

In certain aspects, the filters are commercially available filters(e.g., Semrock Brightline filters). Filters may be specially coated todeal with high output of the light source 22. The specific wavelengthsof the filters, the number of the filters, and other variables may varybased upon, for example, the particular type of sample being imaged, thespecific fluorescence channel(s), and other factors known to those ofskill in the art.

Automated Focusing

In certain aspects, automated focusing includes an automated focusingcomponent, such as that provided by the Nikon TiE Perfect Focus System(PFS) which utilizes the reflection of LED generated long-wavelengthlight (e.g., 870 nm) on the bottom of a sample plate to determine thedistance from the tip of the objective to the bottom of the sampleplate. This allows the system to automatically correct for mechanicaland thermal induced variations in sample plate position. Utilization ofthe long-wavelength LED enables the PFS to be used with a large varietyof fluorophores emitting in the wavelength range between 340 and 750nanometers.

In other aspects, commercial imaging software from Metamorph (UniversalImaging Corporation (UIC)) or customized software provides softwaredrivers that are able to automatically focus the microscope via stagecontroller 40. The drivers send signals to motors that control an X-Ystage position and a Z-axis focus knob 42. Stepper motors may beemployed to automate such movement (not shown). Generally, to focus animage that is collected by the system optics, a fast-Fourier analysis isperformed to measure the spatial frequency of the image, and then thecomputer moves the Z-position and repeats the analysis at a new depth. Alocal maxima in spatial frequency is determined which corresponds to thefocal plane. As referenced above, transmitted light (rather thanepifluorescence images) is preferably used for focusing because reducedlight exposure intervals are required (which limits phototoxicity) andfast acquisition times. The CCD camera resolution can be reduced duringimage acquisition for determination of the focal plane to increasespeed. This provides up to a 100-fold improvement in acquisition speedand substantial reduction in phototoxicity.

Embodiments include the use of both the Perfect Focus System (PFS) andthe Z-plane focusing described above. Such embodiments may be usefulwhen using samples (e.g., multi-well plates) with thin bottoms,especially if a long focal length objective is used. In certainembodiments, PFS may find it difficult and/or impossible to get a‘lock.’ Accordingly, in such embodiments Z-plane focusing may beemployed followed by PFS, thereby enabling PFS to get a ‘lock’ andmaking the system more robust.

Imaging Modalities

In some embodiments, a system as described herein can operate in one orboth of two different imaging modalities: wide-field fluorescentmicroscopy and confocal microscopy. Wide-field microscopy allows a userto collect images in two dimensions at each time point. With theaddition of confocal microscopy, three-dimensional images can becollected at each time point. In some embodiments, for example, aYokogawa CSU-W1 spinning disk confocal can be attached to the right portof a microscope as described herein for collection of three-dimensionalimages (XYZ) at each time point. In such embodiments, the final imagescollected with the system are in four dimensions (XYZT) where T=time.This allows for collection of new features in samples relative toprevious systems. In some embodiments, while utilizing the confocalmodality, six laser lines are used for excitation of fluorophores at sixdifferent wavelengths simultaneously. For example, it is possible toimage live zebrafish embryos, brain slices, and more complex organoidsmade from stem cells or any structure that has a three-dimensionalstructure whereas, with wide-field microscopy alone, the system islimited to two-dimensional structures such as monolayers of cells thatare relatively flat. The CSU-W1 confocal is equipped with 2 scientificCMOS Zyla 4.2 cameras that collect light in two different channelssimultaneously. This hardware, when implemented with suitable software,allows for the collection of faster time-lapse images at each imaginginterval. This allows, for example, for the collection of fasttime-lapse images on a first day, return to, e.g., the same cell on asubsequent day, and collection of another set of fast time-lapse images(e.g., every 10 seconds for 20 minutes) of, e.g., the same cell.

The fast time-lapse imaging capability allows for user-controlledscheduling and time-lapse control with images taken every 10 seconds orless, e.g., every 5 seconds. For example, seven scheduled time-pointscan be taken 24 hours apart, with each time-point including 120 imagestaken every 5 seconds.

A discussed above, 3-D images can be produced with the Yokogawa CSU-W1spinning disk confocal. The Spinning disk performs confocal imaging,removing all out of focal plane light, while also multiplexing theacquisition across the surface of the camera sensor, allowing a highspeed acquisition of single focal planes. When the confocal is used incombination with a high precision piezo Zstage (PZ-2150), these focalplanes can be stepped through the sample, producing a series of imageswhich can then be reconstructed to form a three-dimensional image. Thesefeatures can be incorporated into a system as described herein, allowingseveral time-points, over several days or weeks to be acquired, therebycreating a 4-dimensional image (X,Y,Z,T). By way of example, FIG. 27shows an image acquired utilizing the Yokogawa CSU-W1 spinning diskconfocal in combination with a high precision piezo Zstage (PZ-2150).The large panel shows the image of one focal plane (Nikon CFI Plan ApoLambda 10X) with the thinner panels on the side and bottom showing thecollection of imaging planes collected. The white lines in each panelshow the location of each image plane respectively.

In some embodiments, phase-contrast objectives and filters and amotorized condenser can be integrated into a system as described hereinto collect phase contrast images. This allows for the capture of phasecontrast images of, e.g., the same cells in longitudinal images. By wayof example, the image in FIG. 28 was acquired using a 40× phaseobjective from Nikon (CFI S Plan Fluor ELWD ADM 40×) in combination witha phase 2 aperture (Nikon Ph2) mounted in the motorized condenser. Whenthese two elements are used together, they allow for automated, phasecontrast imaging in the same imaging session as Epifluorescence orconfocal imaging, increasing the amount of information acquired, e.g.,from each well.

Software Control of Peripherals

As described above, a system may include one or more processorsconfigured to control the system and/or subsystem. In certain aspects,the processor(s) may execute instructions from one or more softwaremodules to cause the system or subsystems thereof to facilitate bulksample storage, to remove and/or identify a specific sample from bulksample storage, to place a sample on an imaging device, to align asample on an imaging device, to image a sample, to place a sample intobulk sample storage, and/or to process image(s) taken of a sample.

FIG. 18 presents a block flow diagram of a process for control of asystem of the present disclosure. In this embodiment, one or moreparameter files are read by scheduling software (e.g., custom writtensoftware from BioSero Inc.). The parameter files may be tailored for aspecific sample type, wherein a sample identification subsystem is usedto identify a particular sample, and a processor is configured toidentify the specific parameter file(s) for that sample.

Accordingly, the handling, imaging, and/or image processing may each betailored for a particular sample. Where samples present on multi-wellplates, each well of that multi-well plate may itself have specificparameter file(s) that specify the handling, imaging, and/or imageprocessing for one or more individual wells. For instance, a usage formulti-well plates may involve a drug and/or genetic screen where only asingle condition such as a drug or genetic modifier is altered in eachwell in order to determine its affect on the measured outcome. Thisexperiment may involve identical imaging parameters in each of themeasured wells since it is effectively the same experiment in each well.An alternate usage enabled by systems of the present disclosure would beto use each well or a group of wells to carry out experiments unrelatedto other wells or groups of wells. These experiments may differ in moreways than a screen normally does such as the cell type being measured,the imaging area covered, the biosensors measured, or the frequency ofimaging. The ability to adapt each well to experimental requirements hasseveral advantages, such as (i) increasing the number of experimentswhich can be carried out without the cost of selecting sub-optimalimaging parameters due to the constraints of adjacent experiments; (ii)reducing material costs by reducing the number of plates necessary for agiven number of experiments; and (iii) allowing for experiments to adaptto uncertain and irregular access to cells. For instance, iPS (inducedpluripotent stem) cells require weeks to months to mature to aparticular cell-type. It is common for some wells to be lost due to celldeath or poor differentiation, but it is not easy to predict how many.Being able to carry out several experiments on a single plate allows theexperimenter to adapt the investigation to the number and quality ofavailable wells.

Moreover, in certain aspects timepoint-specific imaging parameters canalso, or instead, be used if there is an initiating optical stimulusrequired for the experiment or if there are different stages/questionbeing asked during the experiment which require different imagingparameters. For instance, an optical-pulse chase experiment whichrelates the rate of turn-over of a photoswitchable protein within a cellto the survival of the cell nicely demonstrates both points. In order tolabel a population of photoswitchable proteins, a pulse ofshort-wavelength light is applied at the first timepoint of theexperiment. The next several timepoints may be carried out with highfrequency and imaging in multiple channels to track the both the celland the amount of photoswitched protein within the cell. The survival ofthe cell is normally carried out with much lower frequency and imagingjust the morphology marker of the cell to track cell death.

Embodiments of systems and methods of the present disclosure includeperforming runtime analysis used to control the handling, imaging,and/or image processing of one or more samples. For instance, in certainembodiments an area is scanned—or images are collected—until a certainnumber of objects have been imaged, where the number of objects may be auser-defined parameter. Analysis of the images during the experimentalrun enables an interactive runtime which can alter imaging parameters inresponse to experimental requirements. Examples of these requirementsinclude, but are not limited to, a certain minimum number of cellsimaged for statistically relevant comparisons between conditions, acertain range of cellular fluorescent intensities which are within thelinear range of the camera, or a time consuming imaging protocol whichwould be impractical to carry out at every position imaged on a platesuch as imaging every second for several minutes to study dynamics orhigh-resolution spectral imaging. In certain aspects, one or moreparameters (e.g., imaging parameters, such as the filter(s) applied orthe frequency of imaging) may be altered when a certain object is imagedand detected.

Turning again to FIG. 18, the scheduling software may interface withintegration software, which is in communication with one or morecomponents of the system. The integration software may include aninterface to the driver(s) for a given component, causing it to take oneor more actions as determined by the parameter files and schedulingsoftware. Further, the integration software may interface with imagingdevice software (e.g., μ-Manager software) to control the imagingdevice. As appreciated by one of skill in the art, the particular meansof connecting software modules (e.g. through an application programminginterface (API)) will be dictated by the particular software modulesemployed. In certain aspects, the means for connecting software modulesmay include a client/server relationship, wherein one module (e.g., thescheduling software) acts as the server, issuing instructions to one ormore clients (e.g., the imaging device software).

In certain aspects, software (e.g., the scheduling software) may be usedto manage runtime conflicts. Runtime conflicts may occur, for example,where a system includes a plurality of samples, a plurality of imagingdevices, a plurality of transport devices, a mix of different sampletypes, and the like. In certain embodiments, the scheduling softwaredetermines the runtime for a sample by start time and/or priority. Forexample, every experiment may be initially programmed with (i) starttime(s) and/or (ii) duration. When an experiment is entered into thescheduling program, such as by a user, a priority may be assigned. Thescheduling software may determine the next sample (e.g., a plate, suchas a multi-well plate) to run based upon the following algorithm:

-   -   1. List the samples to be run in order of start-time. The list        may include samples which have a start-time that has already        past but has not yet been sent to a microscope.    -   2. Using the duration of each experiment, determine which        experiment(s) would conflict at any point from the start to end        of the experiment.    -   3. If an experiment conflicts with another experiment which has        a higher priority, then remove it from consideration as the next        sample to run. Optionally, include a function which increases        the priority of an experiment based on the amount of time which        has elapsed since the scheduled start-time. This ensures that        even low priority samples will eventually be run.    -   4. The first item on the list is then the highest priority        experiment within the context of adjacent experiments that can        fit within the time before the start of another experiment of        greater priority. Run that first experiment.        In other aspects, the scheduling software may also, or instead,        utilize a calendar to determine conflicts. Experiment start        times and duration are mapped to a calendar, such as a calendar        represented as a horizontal line. Experiments which conflict at        any point from the start to end of the experiment are signaled        by placing one of the conflicting experiments on a horizontal        line below the standard line.

In certain aspects, process control includes one or more instructionsthat are issued in parallel. FIG. 19 presents an embodiment in whichinstructions are issued in parallel and/or substantially at the sametime. In this embodiment, the integration software and/or imaging devicesoftware cause a processor(s) to move the microscope stage, change theexcitation filter, change the emission filter, change the polychroic,change the illumination intensity, and change the focus plane of theimaging device at substantially the same time. Because such operationsmay take varying times, the system may pause (e.g., have asynchronization point) to allow for at least the slowest operation to becomplete. After this synchronization point, the system may cause theshutter to be opened (e.g., for a defined period), exposed, and closed.Upon causing the shutter to close, the system may again perform severaloperations in parallel and/or at substantially the same time, includingtransferring the image, move the microscope stage, change the excitationfilter, change the emission filter, change the polychroic, change theillumination intensity, and change the focus plane of the imagingdevice. Once stored, the image may be subjected to additionalprocessing, such as described below.

Systems of the present disclosure may produce a notification/alarm, withsuch notifications or alarms produced using hardware, software, or acombination thereof. For example, systems may include one or moresensors that monitor (e.g., continuously or discretely monitor) factorssuch as system power, environmental control (e.g., temperature, CO₂concentration inside the bulk storage subsystem, etc.), and the like. Toensure continuous power, a system may include one or moreuninterruptible power supplies (UPS) to bridge house power to the system(or a subsystem thereof) and serve as an immediate power source untilbackup generators are started. In the event of a power outage, thesystem may be configured to send notification to responsible parties byany convenient means, such as by text and/or email. Moreover, systemsmay include hardware and/or software configured to monitor systemperformance and health and to store such information non-transiently,such that fluctuations in system conditions (e.g., environmentalcontrol) can be linked to experimental changes observed.

Image Processing

In certain aspects, the systems of the present disclosure may includeone or more processors configured to process an image acquired asdescribed above. As shall be detailed below, such processing may includeone or more of the following: organize the images for a particularsample; stitch two or more images for a particular sample together;align images; identify objects (e.g., cells, such as neurons) within animage; track an object (e.g., a cell, such as a neuron) through atemporal series of images; extract data (e.g., fluorescence data) fromobjects identified within an image; and/or analyze the resulting images.

Systems of the present disclosure may employ a single pipeline approach,in which each subsequent step in the pipeline is acting on the resultingimages from the previous step. Systems may also, or instead, employmultiple pipelines of image analysis, e.g., 2 or more, 3 or more, 4, ormore, 5 or more, or 10 or more. That is, in certain aspects severalpipelines all act on the original image(s) and combine the resultingimage(s) from each pipeline to create a result, such as segmentedregions. In addition or alternatively, these multiple pipelines may alsohave iterative or sequential components such that the information gainedfrom one pipeline is used in the preprocessing step of the same oranother pipeline.

For example, a sample (e.g., a sample that likely includes a cell, suchas a neuron) may be imaged by a plurality of imaging pipelines, wherethe results of each are combined to classify the sample. For instance, asample including a neuron may be imaged using first, second, and thirdpipelines employing first, second, and third imaging systems,respectively, and first, second, and third parameter files,respectively, each pipeline classifies an area as a neuron or not. Thereare several benefits to parallel rather than consecutive pipelines,including (i) each pipeline can have less strict parameters (which helpsto keep the false negative rate low) without adversely increasing thefalse positive rate; (ii) more complex negative filters (in contrast tosimple limits such as area min/max) which detect characteristics thatshould never or rarely be present in the segmented cell type can beused; and (iii) filters which are not particularly sensitive but arereliable indicators of neurons or non-neuronal cells can be used infuzzy combination as described below. FIG. 25, Panels A-B illustratethat such an approach to segmentation, using intensity and spatialsegmentation, results in improved specificity without sacrificingsensitivity.

The individual results from the individual pipelines may be combinedusing any convenient method, such as absolute classification (e.g.,requiring positive identification of an area as a neuron by allpipelines for the object to be classified as a neuron), or fuzzyclassification (e.g., via voting and/or a weighted combination of theresults of the individual pipelines). Such combinations andclassifications may involve applying one or more statistical or learningmachine algorithms, such as genetic algorithms, neural networks, hiddenMarkov models, Bayesian networks, support vector machines, and the like.In certain aspects, segmentation of an image resulting from theaforementioned absolute or fuzzy combination can be done using one of atleast two methods: contours on the final binary image or overlap of thefinal binary image with a segmentation which more fully outlines thedesired cellular morphology. While parallel pipelines may moreaccurately determine whether a cell is of the proper cell type, thecombination may not accurately reflect the desired total area orsub-region of the cell. Thus, a previous or other segmentation such asintensity segmentation can be used as the contour if the area within theintensity segmentation contains a sufficient number of positive pixelsfrom the final combined image.

In certain aspects, image processing includes application of one or moremachine vision algorithms. Examples of machine vision algorithms ofinterest include, but are not limited to, those described in Jain,Ramesh, Rangachar Kasturi, and Brian G. Schunck. Machine vision. Vol. 5.New York: McGraw-Hill, 1995; Sonka, Milan, Vaclav Hlavac, and RogerBoyle. “Image processing, analysis, and machine vision.” (1999); Davies,E. Roy. Machine vision: theory, algorithms, practicalities. MorganKaufmann, 2004; and Hornberg, Alexander, ed. Handbook of machine vision.Wiley-VCH, 2007; the disclosures of which are incorporated herein byreference.

Image Organization

In certain aspects, a processor is configured to execute one or moresoftware modules to organize the image file(s) for a given sample.Software modules may perform one or more of the following organizationtasks: renaming of file names, creation of one or more directories,automated conversion of one file type to a different file type, or up-or down-conversion of the image(s).

In certain aspects, a software module identifies files in a user-definedpath having a user-defined prefix and/or suffix. FIG. 20, Panel Apresents a non-limiting example of a file name having a user-definedprefix and suffix. Here, the image file has been created with a namethat provides information about the date of acquisition, experimentname, timepoint, hour, fluorescence channel, well, montage index, andfile type.

Using the example from FIG. 20, panel A, a software module (e.g., ascript, such as a Perl script, Python script, and the like) may beconfigured to cause a processor to identify all file types in theuser-defined path having the prefix “PID” and/or the suffix “.tif”. Thenames of such files may be parsed, such as by using a regex engine(e.g., a Perl regex engine). Once parsed, the processor may create amulti-tiered file organization structure, such as that illustrated inFIG. 20, panel B. In this example, a directory having the experimentname is created in a user-defined path, if the directory is not alreadycreated. Within that directory, a separate sub-directory may be createdfor each fluorescence channel of the experiment (e.g., “RFP,” “GFP,”etc.). Within each fluorescence channel sub-directory, a sub-directorymay be created for each timepoint that is recorded (e.g., T0, T1, . . .T8). A sub-directory may be created for each timepoint, in whichseparate sub-directories are created for each well(s) of the sample. Forexample, for a 96-well plate, a total of 96 sub-directories may becreated for each timepoint, corresponding to wells A1, A2, . . . H10,H11, and H12. For each well, a separate sub-directory may be created, inwhich the images for each montage index are stored. Alternatively, allimages can be contained in a single root directory and the imagefilenames can be parsed and organized during analysis. In other words,in some embodiments a hierarchical folder structure need not beutilized. Instead, all images can be contained within a single rootfolder.

As described above, systems may be preferably configured to allowimaging of live cells grown on tissue culture plastic that can bemaintained for long lengths of time (days to months) in tissue culturedishes. Accordingly, in some aspects many timepoints may be taken for agiven experiment. Updating a file structure such as that presented inFIG. 20, Panel B may be accomplished by any convenient means. In certainaspects, one or more software modules may perform the following tasksfor a given file of the type presented in FIG. 20, Panel A. Once thefile name is parsed, the software may check whether a directorycorresponding to the Experiment Name exists. If it does not exist, thedirectory is created; otherwise, the system then checks whether asub-directory corresponding to the fluorescence channel exists. If thesub-directory corresponding to the fluorescence channel does not exist,it is created; otherwise, the system checks whether a sub-directorycorresponding to the particular timepoint exists. In most cases, such adirectory will not exist, and must be created. Within thatsub-directory, the system creates a sub-directory for the given wellidentified in the file name (e.g., “A1”), and a correspondingsub-directory within that sub-directory corresponding to the montageindex number. The file is then moved or copied to that sub-directory,with the process repeated for any other files contained within theuser-defined path.

Means for checking whether a directory has already been created, forcreating a directory, for renaming files, and/or for moving files areknown in the art.

Image Stitching

In certain aspects, two or more images may be acquired for a particularsample (e.g., for a particular well). The individual images may bestitched to produce a montage image. FIG. 21, Panels A and B areillustrations depicting general principles of construction of a montageimage. Panel A presents nine images taken of an individual well of amulti-well plate. For each image, the areas indicated by dashed linescorrespond to areas of the image that are substantially identical tothose areas indicated in dashed lines of the immediately adjacentimages. These images are then stitched using one or more softwaremodules to produce a montage image, as depicted in Panel B. As isapparent from FIG. 21, Panel B, the resulting montage image contains oneor more areas where data from two or more individual images contributedto the resulting montage image.

A variety of stitching software modules may be employed in systems ofthe present disclosure. In certain aspects, the modules may performrigid stitching, wherein images are overlaid at specific coordinates. Incertain aspects, the modules may perform flexible stitching, in whichpixel information at an overlap region is used to adjust the position ofthe images so as to achieve an improved alignment. Where flexiblestitching is employed, a primary cell marker channel (e.g., RFP) may beused to calculate coordinates which may be stored (e.g., in memory). Incertain aspects, all subsequent channels (e.g., GFP, etc.) may read thisinformation so there is perfect alignment between the images fromdifferent channels.

An examples of stitching algorithms of interest include, but are notlimited to, those described in S. Preibisch, et al. (2009)Bioinformatics, 25(11):1463-1465; the disclosure of which isincorporated herein by reference. In certain embodiments, software, suchas custom (e.g., Pilotscript) or commercially available software, isused to compute starting positions of images within a montage. Theseimages may then be stitched (e.g., using the command “Stitch Collectionof Images” using the Stitching plugin in ImageJ). In certain aspects,such stitching is performed where the fusion method is set to linearblending, regression=30; max/avg=2.5, and absolute=3.5.

FIG. 21, Panel C, provides a montage image of a single well from a 96multi-well plate wherein a rigid stitching algorithm was utilized toform the montage image.

Image Alignment

In certain aspects, a processor may be used to perform an in silicoimage alignment step before subsequent analysis is performed on theimages (e.g., cell tracking, etc.). This may be required because incertain experiments each image is acquired after several hours or morehave elapsed (e.g., 12-24 hours or more). When working with biologicalsamples, cells may have moved, died, or changed in intensity betweeneach timepoint. Moreover, additional shifts may be introduced bymicroscope stage hysteresis.

In certain aspects, pixel intensities are used to map image coordinatesby x=u+Δu. Each timepoint T(t) is aligned to T(t−1), where t={2 . . .N}. Embodiments include the use of an automatic subpixel registrationalgorithm that minimizes the mean square intensity difference between areference and a test data set, which can be either images(two-dimensional) or volumes (three-dimensional). In certain aspects,image alignment is performed substantially as described in P. Thévenaz,et al. IEEE Transactions on Image Processing, vol. 7, no. 1, pp. 27-41,January 1998; the disclosure of which is incorporated herein byreference.

Embodiments may include the use of one or more software modules fromImageJ (described in Abramoff, et al., (2004) Biophotonics International11, 36-42; the disclosure of which is incorporated herein by reference).In certain aspects, the ImageJ plugins StackReg_.jar, MultiStackReg.jar,and/or TurboReg_.jar are used by a software module to perform imagealignment.

Cell Identification

Aspects of the subject systems may include one or more software modulesto identify particular components (e.g., cells) from images.

A general depiction of the process of labeling objects is presented inFIG. 22, Panel A, which is a block flow diagram of a method for labelingand/or tracking objects (e.g., cells) in an image, such as a montageimage. As depicted in this diagram, in the method 600 an image ispreprocessed 601. Such preprocessing 601 may be iterative, and/orrepeated, such that an image is subjected to one or more preprocessingsteps. The preprocessed image is then analyzed for the detection of oneor more objects (602). A filtering step 603 may be employed, and objectsmay be tracked 604 (e.g., tracked over time).

Any convenient preprocessing, object detection, filtering, or trackingmethods may be employed in practicing the method 600. For example, incertain aspects of methods for cell identification, the method includesa step of flat-field correction, such as by using and/or obtaining aflat-field reference image, a pseudo flat-field reference image, and/orusing a FFT bandpass function. A flat-field reference image may becalculated using any convenient means, such as using median or otherpercentile metrics or fitting and subtracting a model of the backgroundto the image. Suitable flat-field correction techniques include thosedescribed in J A Seibert, et al. Medical Imaging 1998: Physics ofMedical Imaging, 348 (Jul. 24, 1998); the disclosure of which isincorporated herein by reference.

FIG. 22, Panel B depicts a more specific embodiment of such a method600. In this method 700, the steps of creating a montage image 701,enhancing contrast and decreasing bit depth 702, and smoothing 703correspond to preprocessing steps 601, using the example of FIG. 22,Panel A. Similarly, the step of detecting peaks 704 corresponds to thestep of detecting objects 602. The steps of method 700 of eroding anddilating 705 and applying a morphology filter 706 map to the step offiltering 603. Finally, the steps of method 700 of labeling objects 707and tracking objects 708 map to the step of tracking 604, using theexample of FIG. 22, Panel A.

As depicted in FIG. 22, Panel B, in certain aspects an image, such as amontage image, is first subjected to contrast enhancement and decreasedbit depth 702. A smoothing operation 703 may be employed. Next, a peakdetection algorithm 704 is used to identify peaks above a user-definedthreshold in the image histogram. In certain aspects, the threshold isabout 95%, and/or the peak region must be 2 fold greater thansurrounding regions. Peaks are defined as an absolute peak andsurrounding pixels within a defined range (e.g., all 8 adjacent pixels).Next, the image is eroded and dilated 705 to remove thin objects. Forexample, objects having a radius of about 10 pixels or less, such asabout 5 pixels or less, may be removed.

In certain aspects, a morphology filter 706 is applied, such as amorphology filter that eliminates objects based on area, eccentricity,and/or circularity. After applying a morphology filter 706, the objectsmay be labeled 707 and tracked 708. Morphology filters may varydepending upon the particular type of sample being investigated, thespecific hardware and image acquisition parameters, and other variablesknown to those of skill in the art. In certain aspects, objects smallerthan about 800 to 1000 pixels with 20× magnification on a 1392×1040 pxCCD are removed, such as where the sample includes cells such asneurons. Embodiments may also, or instead, utilize a bandpass filter toinclude or exclude objects, where such a filter that is implemented as aFourier transform, a difference of Gaussians, or a neurite/distance map.

Circularity is defined using the equation 4π(area/perimeter²); where avalue of 1.0 indicates a perfect circle, and a value of 0.0 indicates anincreasingly elongated polygon. For a perfect circle, this number isequal to 1. Since this calculation is done in pixels, the measure willnot be very meaningful for very small objects (for example, an objectthat consists of only one pixel will have both the area and theperimeter equal to 1, so circularity will be 4π, even though the objectis relatively close to a circle). As the object gets bigger, the measureimproves. For instance, for a circle of radius 2 the area is 21 and theperimeter is 16 (number of border pixels), and the circularity is 1.03.

In certain aspects, objects that have circularity cutoff of 0.8 orhigher are retained. In certain aspects, objects that have aneccentricity value above 0.1 (e.g., above 0.2) are retained. In certainaspects, objects having an area of less than about 800 pixels arediscarded. Those objects remaining may be labeled.

In certain aspects, one or more filters are employed that are based ondynamic changes within a tracked object over time. For example, ratherthan employing a set cutoff (e.g., a circularity cutoff), a filteremploys a cutoff or range which measures the change between the currentvalue for a feature and one or more prior values. Such a filter may, forinstance, be based on one or more features that indicate cell health(e.g., neurites). By measuring dynamic changes, as opposed to staticfeatures, such filters may facilitate longitudinal experiments whereobjects (e.g., individual neurons) are tracked over time.

Embodiments of the systems and methods of the present disclosure includeimaging of neurons. In such embodiments, cell identification may involvethe characterization of cellular extensions, e.g., neurites, which maybe used to provide information about the functional and/or health statusof a cell, e.g., a neuron. Such characterization can be used to, e.g.,track differences in neurite parameters between conditions, changes inneurite parameters over time, and/or as a segmentation filter toidentify neuronal vs. non-neuronal cells. Such embodiments may involvethe use of neuron-specific methods which look at cell-specific changesin neurite parameters, and/or image-wide methods which look at how thepopulation of visible neurites, whether from cells on or off the image,change.

For example, embodiments may include the use of neuron-specific methods.In some aspects, such methods involve a topological analysis of thenumber, length, and/or width of the branches on a neuron. Thetopological information is extracted from the set of points which makeup the perimeter of the object. For each point, a list of distances toall of the other points along the perimeter is calculated. In somecases, only distances where the length is entirely contained within theobject is calculated and the value is left empty or 0 otherwise. Thisarray can be represented as a two dimensional image referred to as adistance map where the x or y coordinates represent the nth point alongthe perimeter, and the pixel intensity represents the distance from thepoint #x on the list to point #y on the list. The distance map issymmetric along the diagonal so only half of distances may be calculatedand plotted. The distance map analysis thus transforms a morphologicalanalysis into an intensity analysis. Thresholding the image to eliminateall pixels which represent distances greater than 20-30 pixels (˜6-10microns) removes portions of the cell which would be too wide toclassify as a neurite. Counting the number of objects which remain afterthresholding reflects the number of branches on the neuron. The lengthof the neurite segment is the length of the “valley” remaining aftersegmentation, starting with the point closest to the line of diagonalsymmetry and ending with the point farthest away. Connections betweensegments can be determined by finding neurite segments where theendpoint matches vertically or horizontally with another endpoint on aneurite segment. This approach can be used to quantify neurites in anautomated fashion (see, e.g., FIG. 26, Panels A-E).

Further, counting the number of neurites in neuronal cultures can be anearly and sensitive measure of cell health. Neurite retraction oftenprecedes neuron death when cells are coping with neurodegenerativedisease causing proteins. One way to detect neurites with lowcomputational complexity is to quantify the total area of neurites in animage. A line detector may be used to identify pixels in an image thathave local structures that represent a line. The determination is basedon the second order pixel data (Hessian matrix), which is a measure oflocal intensity curvature. In certain aspects, such an approach uses thefollowing parameters:

1. Sigma start=1 (approximates expected line width)

2. Sigma step=1 (approximates expected line width)

3. Sigma count=1 (approximates expected line width)

4. Beta1=0.5 (sensitivity to blobness)

5. Beta2=25 (sensitivity to second order structure)

The output of the line detector is a binary image with pixels that arepart of lines having a value of 1 and all other pixels having a value of0. The pixels of this binary image are then summed to quantify the totalarea of neurites. For example, FIG. 24 is graph showing autophagyinduction mitigates neurite degeneration induced by a disease model ofamyotrophic lateral sclerosis (TDP43 M337V). Primary neurons weretransfected with GFP as a control or TDP43 M337V. The disease modelneurons were treated with fluphenazine (0.1 μM) or vehicle to determinethe whether autophagy could rescue the disease model phenotype. Imageswere collected every 24 hours with the robotic microscope and neuriteswere quantified using automated analysis as described above.

In certain aspects, one or more of the above steps may be performedusing commercially available software, such as the advanced imagingtoolbox from Pipeline Pilot (Accelrys), ImageJ, Matlab, Perkin ElmerVelocity, Media Cybernetics ImagePro Plus, Metamorph, and/or NikonElements. In other aspects, one or more steps of the process areperformed by custom software modules.

Cell Tracking

As describe above, in certain aspects the systems are configured toallow for precise return to and re-imaging of the same field (e.g., thesame cell) that has been previously imaged. Accordingly, in certainembodiments cells may be tracked across different time points using oneor more software modules.

At a given time point, objects may be identified using the algorithmsand/or modules as described above. For a given time point, once objectsare identified they may be ‘tracked’ by comparing the position,intensity, size, circularity, etc. of the object at a prior and/orsubsequent time point.

Cell-tracking algorithms can be categorized into a number of differenttypes. For example, the centroid cell-tracking algorithm calculates thecenter-of-mass (centroid) of the object of interest. It performs bestwhen all objects move in exactly the same way between consecutiveframes, relative to each other and irrespective of changes in theirshapes and intensities. In contrast, the Gaussian cell-trackingalgorithm directly fits Gaussian curves to the intensity profile andperforms best when the intensities of the objects are same betweenconsecutive frames, even if their movement is random relative to eachother. Another type is a cross-correlation algorithm, which compares animage to a matrix of pixels (user defined) of a successive image. Thematrix may be shifted relative to the image in 1-pixel increments. Foreach increment, a correlation value is calculated that describes howwell the values in the matrix match those of the image, and the programdetermines the shift that yields the maximum correlation value. Thisalgorithm is computationally more intensive than the other two andsignificantly slows the analysis, depending on the size of the matrixselected.

In certain aspects, cells may be tracked based on a strict overlap,wherein if two segmented objects overlap in consecutive images, then theobject in the latter image is relabeled to the same label in the formerimage. In other aspects, cells are tracked by minimumcentroid-to-centroid distance, and/or minimum border-to-border distance.Aspects may include the use of a maximum velocity value, which specifiesthe maximum allowed displacement for a cell in consecutive images. Invarious aspects, any of such cell-tracking algorithms described hereinmay be employed in the systems and methods of the present disclosure.

In certain aspects, objects are tracked between a time point T₁ and alater time point T₂ by identifying the objects present in the respectiveimages (e.g., using a cell identification software module, as describedabove). Data recorded for each object present at T₁ may be recorded,such as the well number, the object index, the time point, the position,intensity, size, circularity, etc. of the object. Likewise, datarecorded for each object present at T₂ may be recorded, such as the wellnumber, the object index, the time point, the position, intensity, size,circularity, etc. of the object. Objects present at T₂ that were notpresent at T₁ may be handled in a variety of ways. In certain aspects,the newly appearing object may be assigned an object numbercorresponding to the next highest available number.

Tracking may be facilitated by using one or more commercially availablesoftware modules, or custom software modules. In certain aspects, cellsmay be tracked by using, for example, Pipeline Pilot (Accelrys) and/orImageJ.

Data Extraction

A number of features may be extracted from the aforementioned images,including morphology features, intensity features, texture features,location features, and the like. In certain aspects, the features thatare extracted correspond to a particular object, such as a cell (e.g., aneuron). Embodiments of the systems and methods of the presentdisclosure also, or instead, include calculating one or more featuresthat take into account one or more neighbors of an object (e.g.,neighboring cells), as is described more fully below.

Morphology features of interest include, but are not limited to, area,perimeter, circularity, convex hull, roughness, and topologicaldescriptors. Intensity features of interest include, but are not limitedto, mean intensity, intensity order statistics, standard deviation,skewness, and kurtosis. Texture features of interest include, but arenot limited to, entropy, neighborhood intensities, homogeneity, andLaw's descriptors (e.g., Level, Edge, Spot, Wave, Ripple). Locationfeatures of interest include, but are not limited to, centroid, boundingbox, and edge touching.

In certain aspects, one or more region shape statistics are extracted,with such region shape statistics of interest including, but not limitedto: equivalent diameter (diameter of the circle of the same area);convex area (area of the convex hull); extent (area divided by the areaof the bounding box); solidity (area divided by the area of the convexhull); Euler number (number of holes subtracted from the number ofconnected blobs); form factor (measure of circularity, where circularityis equal to 4π(area/perimeter²)); nearest neighbor (centroid-centroid);nearest neighbor (boundary-boundary); smallest bounding rectangleparameters, such as orientation (angle between the x-axis and the longerside of the smallest rectangle that contains the whole object), length(longer side of the smallest rectangle that contains the whole object),width (shorter side of the smallest rectangle that contains the wholeobject), and center (X and Y coordinates of the center of the smallestrectangle that contains the whole object); fitted ellipse parameters(filled or contour), such as eccentricity (measure of ellipseelongation; distance from the center to either focus, divided by thelength of major semi-axis), orientation (angle between x-axis and themajor axis in radians), major axis length, and minor axis length.

In certain aspects, one or more region pixel intensity statistics areextracted, with such region pixel intensity statistics of interestincluding, but not limited to, mean; range (difference between themaximum and the minimum values); variance (variance of pixel values);mean absolute deviation (average absolute difference between the pixelvalues and the mean pixel value); standard deviation (standard deviationof pixel values); skewness of pixel values; kurtosis of pixel values;sum of pixel values; sum squared of pixel values; entropy of pixelvalues; center of mass (location of the center of mass); massdisplacement (distance between the geometric center and the center ofmass); spatial moments (spatial moments of orders up to 3); centralmoments (central moments of orders up to 3); normalized central moments(normalized central moments of orders up to 3); Hu moments (Hu momentsof orders up to 3); order statistics, such as intensity order statisticsfor percentages specified in Order Statistics Percentages, suchstatistics derived from the co-occurrence matrix (CM) including energy(sum of squared elements of the CM), contrast (measure of contrastbetween a pixel and its neighbor over the whole object), correlation(measure of how correlated a pixel is to its neighbor over the wholeobject), homogeneity (measure of closeness of the CM to the CM of ahomogeneous region), and entropy (entropy of the CM elements).

Features of interest may be extracted from an individual object and/or apopulation. Populations of interest include, but are not limited to,populations defined as all objects that are within a certain distancefrom a specified object. For example, where an object of interest is acell, the neighbor(s) of that cell can be identified to define apopulation consisting of the cell of interest, along with itsneighbor(s). Accordingly, such an approach facilitates theidentification of cell non-autonomous effects by including parameters ofneighboring cells as variables in determining a cell's fate. Thesenon-autonomous effects could arise from cell-to-cell interaction such asa neural circuit or the release of molecules from a cell such as growthfactors. Depending on the experiment, all or a subset of the cellsvisible in an image may have neighbor analysis applied. In addition, thecells for which neighbor analysis is applied and the cells which areconsidered neighbors may be drawn from the same set or from distinctsets. The distinction between the two populations may be based onparameters computed from the images such as inclusion bodies ormorphology, or they may be based upon experimental manipulation such ascell-specific reporters or transfection with different markers at twotimepoints to label separate populations.

The cells which are considered adjacent can be determined using anyconvenient method. For example, in a first method, all other cells inthe image are ranked by distance from closest to farthest, and apre-determined number of cells from the top of the list are classifiedas neighbors. In a second method, all cells within a certain maximumdistance and possibly further than a certain minimum distance areclassified as neighbors. In a third method, only cells which havephysical connections where one part of a cell is in apposition withanother cell are classified as neighbors. Apposition can be determinedusing morphology or through the use of fluorescent markers whichlocalize to cellular junctions. These methods are not exclusive soseveral may be applied to the same dataset. In all three methods, cellswhich are within a certain distance of the edge can be excluding fromanalysis due to incomplete data on their neighbors.

Such neighbor analysis may be used to compute a number of variables.Variables which are used in determining a cell fate in neighbor analysiscan include the normally calculated cellular parameters for eachneighboring cell as well as aggregate data such as the cumulative,average, or median of any cellular parameter where neighboring cells aredefined using one of the above methods. The center of the cell can bedetermined using at least three different methods. For example, in afirst method, the center is defined as the average of the x and ycoordinates of all the pixels within the cell. In a second method, thecenter is defined as the weighted average of all the cell's pixels wherethe pixel intensity is used as the weight. In a third method, asubcellular region of the cell such as the nucleus is used to calculatethe center using either the average or weighted average.

The end result of extraction is typically an extensive set of features,commonly called a feature vector. Data that is extracted may be exported(e.g., as comma-separated values) to programs like Microsoft Excel orstatistical packages like R for advanced analysis.

In certain embodiments, such data may be analyzed using one or moremachine or statistical learning algorithms to facilitate theidentification of relationship(s) between one or more features and astate, such as a disease state. Examples of machine learning algorithmsof interest include, but are not limited to, AODE; artificial neuralnetworks; backpropagation; Bayesian statistics; Naive Bayes classifier;Bayesian network; Bayesian knowledge base; Case-based reasoning;Decision trees; Inductive logic programming; Gaussian processregression; Learning Vector Quantization; Instance-based learning;Nearest Neighbor Algorithm; Analogical modeling; Probably approximatelycorrect learning (PAC) learning; Symbolic machine learning algorithms;Sub symbolic machine learning algorithms; Support vector machines;Random Forests; Ensembles of classifiers; Regression analysis;Information fuzzy networks (IFN); Linear classifiers; Fisher's lineardiscriminant; Logistic regression; Quadratic classifiers; k-nearestneighbor; C4.5; Hidden Markov models; Data clustering;Expectation-maximization algorithm; Self-organizing maps; Radial basisfunction network; Vector Quantization; Generative topographic map; Apriori algorithm; Eclat algorithm; FP-growth algorithm; Hierarchicalclustering; Single-linkage clustering; Conceptual clustering;Partitional clustering; K-means algorithm; Fuzzy clustering, dynamicBayesian networks; and the like.

Exemplary Embodiments

Non-limiting exemplary embodiments of the present disclosure areprovided as follows:

-   1. An imaging system, the system including:

an imaging device including a sample holder;

a transport device configured to place a sample in the sample holder;

a processor in communication with the imaging device and the transportdevice; and

memory operably coupled to the processor, wherein the memory includesinstructions stored thereon for acquiring an image of the sample,wherein the instructions, when executed by the processor, cause theprocessor to:

-   -   move the sample via the transport device to the sample holder of        the imaging device;    -   identify a fiduciary mark on the sample using the imaging        device;    -   move the sample holder so that the fiduciary mark is in        substantially the same position as in a reference image; and    -   acquire an image of the sample using the imaging device.

-   2. The system according to 1, wherein the sample includes a plate.

-   3. The system according to 1, wherein the plate is a multi-well    plate.

-   4. The system according to 3, wherein the multi-well plate includes    about 96 wells or more.

-   5. The system according to any of 2-4, wherein the plate includes    plastic.

-   6. The system according to 5, wherein the plate is black.

-   7. The system according to any of 1-6, further including a bulk    sample storage subsystem.

-   8. The system according to 7, wherein the transport device is    configured to move the sample from the bulk sample storage subsystem    to the sample holder of the imaging device.

-   9. The system according to 7 or 8, wherein the instructions, when    executed by the processor, cause the processor to cause the    transport device to move the sample from the bulk sample storage    subsystem to the sample holder of the imaging device.

-   10. The system according to any of 7-9, wherein the transport device    is configured to move the sample from the sample holder of the    imaging device to the bulk sample storage subsystem.

-   11. The system according to any of 7-10, wherein the instructions,    when executed by the processor, cause the processor to cause the    transport device to move the sample from the sample holder of the    imaging device to the bulk sample storage subsystem.

-   12. The system according to any of 7-11, wherein the bulk sample    storage subsystem is configured to store 5 or more samples.

-   13. The system according to any of 7-12, wherein the bulk sample    storage subsystem is configured to store 20 or more samples.

-   14. The system according to any of 7-13, wherein the bulk sample    storage subsystem includes a heating element configured to maintain    the bulk sample storage subsystem at a specified temperature.

-   15. The system according to any of 7-14, wherein the bulk sample    storage subsystem includes a cooling element configured to maintain    the bulk sample storage subsystem at a specified temperature.

-   16. The system according to any of 7-15, wherein the bulk sample    storage subsystem includes a robotic arm configured to transfer a    sample from the bulk sample storage subsystem to the transport    device.

-   17. The system according to any of 7-16, wherein the bulk sample    storage subsystem includes a robotic arm configured to transfer a    sample from the transport device to the bulk sample storage    subsystem.

-   18. The system according to any of 1-17, wherein the transport    device is a robotic arm.

-   19. The system according to 18, wherein the robotic arm includes a    plurality of grippers configured to engage the sample.

-   20. The system according to 19, wherein the grippers exert a lateral    pressure on the sample.

-   21. The system according to 18 or 19, wherein the grippers include    adjustable elements for engaging samples of different sizes.

-   22. The system according to 21, wherein the adjustable elements are    manually adjustable.

-   23. The system according to any of 1-22, further including a sample    identification subsystem.

-   24. The system according to 23, wherein the sample identification    subsystem includes a barcode reader configured to read a barcode on    the plate.

-   25. The system according to 23 or 24, wherein the sample    identification subsystem is in electronic communication with a    processor configured to identify the sample.

-   26. The system according to 25, wherein the processor is configured    to tailor the image acquisition steps for the sample.

-   27. The system according to any of 1-26, wherein acquiring an image    of the sample includes deconvolving a multi-wavelength image into    its component wavelengths.

-   28. The system according to any of 1-27, wherein the imaging device    includes an inverted microscope body.

-   29. The system according to 28, wherein the sample is imaged from    below.

-   30. The system according to any of 1-29, wherein the fiduciary mark    is located on the bottom of the sample.

-   31. The system according to any of 1-30, wherein the sample holder    is attached to a microscope stage of the imaging device.

-   32. The system according to any of 1-31, wherein the sample holder    is removable from the imaging device.

-   33. The system according to any of 1-32, wherein the sample holder    includes at least two walls defining a cutout portion, an internal    beveled edge, and an internal bottom lip portion.

-   34. The system according to 33, wherein the internal beveled edge    comprises an angle relative to a plane of the bottom lip portion of    the sample holder that is from about 85 deg. to about 25 deg.

-   35. The system according to 34, wherein the angle is from about 70    deg. to about 40 deg.

-   36. The system according to any of 1-35, wherein the sample holder    is sized and shaped to receive a sample having a 127.5 mm×85 mm    footprint.

-   37. The system according to any of 1-36, wherein the sample holder    includes a sample receiving area including at least one corner and    an actuator configured to bias the sample into the at least one    corner.

-   38. The system according to any of 1-17, wherein the imaging device    includes a camera having an exposure of 30 ms or less.

-   39. The system according to any of 1-38, wherein the imaging device    includes a camera having a sensor area of about 170 mm² to about 250    mm².

-   40. The system according to any of 1-39, wherein the camera is an    EMCCD camera.

-   41. The system according to any of 1-40, wherein the imaging device    includes a Xenon light source.

-   42. The system according to any of 1-41, wherein the imaging device    includes a filter wheel.

-   43. The system according to any of 1-42, wherein the memory operably    coupled to the processor includes instructions stored thereon that,    when executed by the processor, cause the processor to:

acquire an image of the sample using the imaging device;

identify a fiduciary mark in the image;

compare the image of the fiduciary mark with a reference image; and

move the sample so that the fiduciary mark is in substantially the sameposition as in the reference image.

-   44. The system according to 43, wherein the comparison of the image    of the fiduciary mark with the reference image includes a    scale-invariant feature transform algorithm.-   45. The system according to any of 1-44, wherein the imaging device    includes an automated focusing component.-   46. The system according to any of 1-45, wherein the memory operably    coupled to the processor includes instructions stored thereon that,    when executed by the processor,

cause the processor to perform at least one action selected from:

organize a plurality of images for a particular sample;

stitch two or more images for a particular sample together;

align two or more images of a particular sample;

identify objects within an image of a sample;

track an object through a temporal series of images; and

extract data from objects identified within an image.

-   47. The system according to any of 1-46, including a second    processor in communication with the imaging device; and memory    operably coupled to the second processor, wherein the memory    includes instructions stored thereon for processing an image of the    sample, wherein the instructions, when executed by the second    processor, cause the second processor to perform at least one action    selected from:

organize a plurality of images for a particular sample;

stitch two or more images for a particular sample together;

align two or more images of a particular sample;

identify objects within an image of a sample;

track an object through a temporal series of images; and

extract data from objects identified within an image.

-   48. The system according to any of 1-47, wherein the sample includes    biological material.-   49. The system according to any of 1-48, wherein the sample includes    one or more cells.-   50. The system according to 49, wherein the one or more cells are    neurons.-   51. An imaging system, the system including:

an imaging device; and

a robotic arm configured to automatically retrieve a sample from a firstsurface and place the sample on the imaging device,

wherein the system is configured to automatically identify a fiduciarymark on the sample, move the sample so that the fiduciary mark is insubstantially the same position as in a reference image, and acquire animage of the sample.

-   52. The system according to 51, wherein the robotic arm is    configured to automatically retrieve the sample from the imaging    device and place the sample on a second surface.-   53. The system according to 51 or 52, wherein the imaging device    includes a sample holder.-   54. The system according to any of 51-53, wherein the sample    includes a plate.-   55. The system according to 54, wherein the plate is a multi-well    plate.-   56. The system according to 55, wherein the multi-well plate    includes about 96 wells or more.-   57. The system according to any of 54-56, wherein the plate includes    plastic.-   58. The system according to 57, wherein the plate is black.-   59. The system according to any of 51-58, further including a bulk    sample storage subsystem.-   60. The system according to 59, wherein the first surface is    contained within the bulk sample storage subsystem.-   61. The system according to any of 59-60, wherein the bulk sample    storage subsystem is configured to store 5 or more samples.-   62. The system according to any of 59-61, wherein the bulk sample    storage subsystem is configured to store 20 or more samples.-   63. The system according to any of 59-62, wherein the bulk sample    storage subsystem includes a heating element configured to maintain    the bulk sample storage subsystem at a specified temperature.-   64. The system according to any of 59-63, wherein the bulk sample    storage subsystem includes a cooling element configured to maintain    the bulk sample storage subsystem at a specified temperature.-   65. The system according to any of 59-64, wherein the bulk sample    storage subsystem includes a robotic arm configured to transfer a    sample from the bulk sample storage subsystem to the first surface.-   66. The system according to any of 51-65, wherein the bulk sample    storage subsystem includes a robotic arm configured to transfer a    sample from the second surface to the bulk sample storage subsystem.-   67. The system according to any of 51-66, wherein the robotic arm    includes a plurality of grippers configured to engage the sample.-   68. The system according to 67, wherein the grippers exert a lateral    pressure on the sample.-   69. The system according to 67 or 68, wherein the grippers include    adjustable elements for engaging samples of different sizes.-   70. The system according to 69, wherein the adjustable elements are    manually adjustable.-   71. The system according to any of 51-70, further including a sample    identification subsystem.-   72. The system according to 71, wherein the sample identification    subsystem includes a barcode reader configured to read a barcode on    the plate.-   73. The system according to 71 or 72, wherein the sample    identification subsystem is in electronic communication with a    processor configured to identify the sample.-   74. The system according to 73, wherein the processor is configured    to tailor the image acquisition steps for the sample.-   75. The system according to any of 51-74, wherein the imaging device    includes an inverted microscope body.-   76. The system according to 75, wherein the sample is imaged from    below.-   77. The system according to any of 51-76, wherein the fiduciary mark    is located on the bottom of the sample.-   78. The system according to any of 53-77, wherein the sample holder    is attached to a microscope stage of the imaging device.-   79. The system according to any of 53-78, wherein the sample holder    is removable from the imaging device.-   80. The system according to any of 53-79, wherein the sample holder    includes at least two walls defining a cutout portion, an internal    beveled edge, and an internal bottom lip portion.-   81. The system according to 80, wherein the internal beveled edge    includes an angle relative to a plane of the bottom lip portion of    the sample holder that is from about 85 deg. to about 25 deg.-   82. The system according to 81, wherein the angle is from about 70    deg. to about 40 deg.-   83. The system according to any of 53-82, wherein the sample holder    is sized and shaped to receive a sample having a 127.5 mm×85 mm    footprint.-   84. The system according to any of 53-83, wherein the sample holder    includes a sample receiving area including at least one corner and    an actuator configured to bias the sample into the at least one    corner.-   85. The system according to any of 51-84, wherein the imaging device    includes a camera having an exposure of 30 ms or less.-   86. The system according to any of 51-85, wherein the camera is an    EMCCD camera.-   87. The system according to any of 51-86, wherein the imaging device    includes a Xenon light source.-   88. The system according to any of 51-87, wherein acquiring an image    of the sample includes deconvolving a multi-wavelength image into    its component wavelengths.-   89. The system according to any of 51-88, further including a    processor in communication with the imaging device and the robotic    arm device; and memory operably coupled to the processor, wherein    the memory operably coupled to the processor includes instructions    stored thereon that, when executed by the processor, cause the    processor to:

acquire an image of the sample using the imaging device;

identify a fiduciary mark in the image;

compare the image of the fiduciary mark with a reference image; and

move the sample so that the fiduciary mark is in substantially the sameposition as in the reference image.

-   90. The system according to 89, wherein the comparison of the image    of the fiduciary mark with the reference image includes a    scale-invariant feature transform algorithm.-   91. The system according to any of 51-90, wherein the imaging device    includes an automated focusing component.-   92. The system according to any of 51-91, further including a    processor in communication with the imaging device; and memory    operably coupled to the processor, wherein the memory operably    coupled to the processor includes instructions stored thereon that,    when executed by the processor, cause the processor to perform at    least one action selected from:

organize a plurality of images for a particular sample;

stitch two or more images for a particular sample together;

align two or more images of a particular sample;

identify objects within an image of a sample;

track an object through a temporal series of images; and

extract data from objects identified within an image.

-   93. The system according to any of 51-92, wherein the sample    includes biological material.-   94. The system according to any of 51-93, wherein the sample    includes one or more cells.-   95. The system according to 94, wherein the one or more cells are    neurons.-   96. A sample holding device, the device including:

two first walls of approximately equal length positioned in opposition,the first walls each defining a cutout portion, an internal beveled edgeand an internal bottom lip portion; and

two second walls of approximately equal length positioned in opposition,the second walls each defining an internal beveled edge and an internalbottom lip portion;

wherein each of the two second walls are shorter in length than each ofthe two first walls, and

wherein the two first walls and the two second walls together define asample receiving area.

-   97. The device of 96, wherein the device is so dimensioned as to    receive a sample having a 127.5 mm×85 mm footprint:-   98. The device according to 96 or 97, wherein the internal beveled    edges of the first walls include an angle relative to a plane of the    bottom lip portion of the first walls that is from about 85 deg. to    about 25 deg.-   99. The device according to 98, wherein the angle is from about 70    deg. to about 40 deg.-   100. The device according to any of 96-99, wherein the internal    beveled edges of the second walls include an angle relative to a    plane of the bottom lip portion of the second walls that is from    about 85 deg. to about 25 deg.-   101. The device according to 100, wherein the angle is from about 70    deg. to about 40 deg.-   102. The device according to any of 96-101, wherein the device is    configured to attach to an imaging device.-   103. The device according to any of 96-102, wherein the device    includes an actuator configured to secure a sample placed in the    device.-   104. The device according to any of 96-103, wherein the device    includes aluminum.-   105. A computer-implemented method of acquiring an image of a    sample, the method including:

moving the sample using a transport device controlled by a processor toa sample holder of an imaging device;

identifying, with the processor, a fiduciary mark on the sample;

aligning, with the processor, the sample holder so that the fiduciarymark is in substantially the same position as in a reference image; and

acquiring, using the imaging device controlled by the processor, animage of the sample.

-   106. The method according to 105, further including moving the    sample, using the transport device controlled by the processor, from    the imaging device to a first surface.-   107. The method according to 105 or 106, further including    processing, with the processor, at least one image of the sample.-   108. The method according to 107, wherein processing includes at    least one action selected from:

organizing a plurality of images of the sample;

stitching two or more images of the sample together;

aligning two or more images of the sample;

identifying objects within an image of the sample;

tracking an object through a temporal series of images of the sample;

extracting data from objects identified within an image; and

analyzing an image.

-   109. The method according to 108, wherein stitching two or more    images of the sample together includes flexible stitching.-   110. The method according to 108, wherein aligning two or more    images of the sample includes minimizing the mean square intensity    difference between a reference and a test data set.-   111. The method according to 108, wherein identifying objects within    an image of the sample includes identifying one or more cells.-   112. A computer-implemented method of identifying objects in an    image of a sample, the method including:

enhancing, with a processor, the image contrast;

decreasing, with the processor, the image bit depth;

smoothing, with the processor, the image;

detecting, with the processor, peaks in the image;

eroding, with the processor, the image;

dilating, with the processor, the image; and

applying, with the processor, a morphology filter to identify objects inthe image.

-   113. The method according to 109, wherein the morphology filter    includes a circularity filter and a size filter.

EXAMPLES

As can be appreciated from the disclosure provided above, the presentdisclosure has a wide variety of applications. Accordingly, thefollowing examples are put forth so as to provide those of ordinaryskill in the art with a complete disclosure and description of how tomake and use the present invention, and are not intended to limit thescope of what the inventors regard as their invention nor are theyintended to represent that the experiments below are all or the onlyexperiments performed. Those of skill in the art will readily recognizea variety of noncritical parameters that could be changed or modified toyield essentially similar results. Thus, the following examples are putforth so as to provide those of ordinary skill in the art with acomplete disclosure and description of how to make and use the presentinvention, and are not intended to limit the scope of what the inventorsregard as their invention nor are they intended to represent that theexperiments below are all or the only experiments performed. Effortshave been made to ensure accuracy with respect to numbers used (e.g.amounts, temperature, etc.) but some experimental errors and deviationsshould be accounted for.

Example 1 Implementation of an Automated Robotic Microscope System

An automated robotic microscope system was developed that was configuredto reduce user intervention relative to existing technologies, and allowfor precise return to and re-imaging of the same field (e.g., the samecell) that has been previously imaged.

In this particular embodiment, the system was configured primarily toimage cells on multi-well plates, such as a Midsci 96-well plate. Platescontaining biological material (e.g., neurons) are stored using anSTX44-ICBT 70 deg. C. incubator, equipped with a Transfer Nest™ (LiCONiCInstruments, Liconic US, Inc., Woburn, Mass.). Adjacent to the TransferNest™ was positioned a Metrologic MS7120 ORBIT barcode scanner,configured to read barcodes printed on the side of the Midsci plates.

Plates positioned on the Transfer Nest™ could be moved to a plate holderon an imaging device using a KiNEDx KX-300-250 robotic arm (PeakRobotics, Colorado Springs, Colo.), equipped with a plate gripperincluding first and second gripper arms. Custom extensions weremanufactured to allow the robotic arm to interact with the plates (FIGS.5-6).

The KiNEDx KX-300-250 robotic arm was used to transfer plates to acustom manufactured plate holder. The plate holder was ground fromaluminum, having a shape as depicted in FIGS. 7-10. Subsequently, acustom plate holder was manufactured to custom specifications by AppliedScientific Instrumentation (Eugene, Oreg.), with a cutout for anelectronic actuator (as depicted in FIGS. 10-17). The plate holder wassecured to the stage of a Eclipse Ti-EB inverted research microscope(Nikon Instruments Inc., Melville N.Y.). To the microscope body was alsoattached an Andor EMCCD iXon3 888 CCD camera (Andor Technology, Belfast,Northern Ireland); Lambda XL light source with an integrated 10-Bcontroller for filter wheel and Smartshutter (Sutter Instrument Co.);and Nikon TiE Perfect Focus System (Nikon Instruments Inc., MelvilleN.Y.). Optical filters were obtained from Semrock (Rochester, N.Y.).

The system employed several software modules. Scheduling was handled byGreen Button Go (BioSero Inc.), using a custom plugin to interface withμ-Manager software (distributed by UCSF, San Francisco, Calif.). Imagestitching and alignment modules incorporated ImageJ (described inAbramoff, et al., (2004) Biophotonics International 11, 36-42; thedisclosure of which is incorporated herein by reference) pluginsStitching jar (described in S. Preibisch, et al. (2009) Bioinformatics,25(11):1463-1465; the disclosure of which is incorporated herein byreference), StackReg_.jar, and TurboReg_.jar. Software modules for cellidentification and tracking included Pipeline Pilot (Accelrys), ImageJ,and custom software.

This system allows for precise return to and re-imaging of the samefield (e.g., the same cell) that has been previously imaged. Thiscapability enables experiments and testing of hypotheses that deal withcausality over time. FIG. 23 provides one such example. This exampleprovides images of one of several primary cortical neurons that weretransfected with two plasmids: EGFP and a new mitophagy reporterconstruct MitoEOS2. The FITC (green) channel (top row) shows themorphology of the neuron which can be used as a mask for determiningsignal intensity but can also be used for additional image analysisroutines such as analysis of neurites as a readout of neuron health. Thefluorescence of the MitoEOS2 construct can be irreversibly shifted fromgreen to red upon illumination with blue light. The RFP images (bottomrow) show the same neuron shown in in the top row of images red-shiftedby exposure to a pulse of blue light at the beginning of imaging. Thesame neuron was unaged eleven times with the first seven images takenevery four hours and the last four images separated by twenty fourhours. The top and bottom rows are images of the same neuron at T1, T2,T3, T4, T5, and T6, wherein T2 is 20 hr after T1, T3 is 24 hours afterT1, T4 is 48 hr after T1, T5 is 72 hr after T1, and T6 is 96 hours afterT1. This figure thus demonstrates the ability of systems of the presentdisclosure to enable experiments and testing of hypotheses that dealwith causality over extended time periods.

Example 2 Selection of Multi-Well Plates

In certain aspects, multi-well plates may be used to grow, store, and/orobserve biological materials using systems of the present disclosure.The impact of plate type was analyzed as follows.

A series of plates were acquired for testing, including: Corning 96-wellpre-coated with Poly-D-Lysine (PDL) (Cat #3372); BD BioCoat 96-wellpre-coated with PDL (Cat #354640); BD BioCoat 96-well pre-coated withPDL-Laminin (L) (Cat #354596); Nunc MicroWell pre-coated with PDL (Cat#152039); Corning Special Optics 96-well (Cat #CLS3614); BD Optilux96-well (Cat #353948); Nunc Optical Bottom 96-well (Cat #165305); Midsci96-well (Cat #TP92096); Nunc 96-well Coverglass bottom (Cat #265300);Matek 96-well glass bottom (Cat #PG96G-1.5-5-F—no PDL); Matek 96-wellglass bottom (Cat #PG96G-1.5-5-F—with PDL); IBIDI 96-well u-plate (Cat#89626). Corning (Cat #3596) plates were used as control plates. Platesthat were not pre-coated were coated with Poly-D-Lysine (50ug/ml-Millipore Cat #A-003-E) and Laminin (5 ug/ml—Sigma Cat#L2020-1MG). Plates were left overnight at 37° C. with the coatingmedia, followed by two sterile water washes.

All plates were tested at the same time. Plates were coated using thecoating media from the same pool and were plated using the neurons alsofrom the same pool.

Survival of primary mouse neurons: To test how well primary mouseneurons survived on each plate relative to control, each well was imagedonce in brightfield on DIV 3, 10, 20, 30 and 40. The number of cells ineach image was averaged to give the mean and SD for each plate. This wasthen plotted against the DIV for each plate. Plates that showedsignificant survival difference for primary mouse neurons when comparedto the control plates were: BD BioCoat 96-well pre-coated with PDL (Cat#354640); BD BioCoat 96-well pre-coated with PDL-L (Cat #354596);Corning Special Optics 96-well (Cat #CLS3614); Nunc 96-well Coverglassbottom (Cat #265300); and Matek 96-well glass bottom (Cat#PG96G-1.5-5-F—with PDL).

Number of images in ‘perfect’ focus: Plates may be imaged with thebottom side facing the optics of a microscope (FIG. 4). Plates weretested to determine which could be imaged in this manner to give imagesin sharp focus. The number of images in sharp focus was counted by eyefor each plate for each DIV image stack. The number of focused imagesper plate was plotted against the DIV. Plates Corning Special Optics96-well (Cat #CLS3614); Nunc Optical Bottom 96-well (Cat #165305);Midsci 96-well (Cat #TP92096); Nunc 96-well Coverglass bottom (Cat#265300); Matek 96-well glass bottom (Cat #PG96G-1.5-5-F—no PDL); andMatek 96-well glass bottom (Cat #PG96G-1.5-5-F—with PDL) gave 92+(96%)focused wells in each plate.

Plates in which 95 wells out of the 96 were in sharp focus were Midsci96-well (Cat #TP92096); Nunc 96-well Coverglass bottom (Cat #265300);Matek 96-well glass bottom (Cat #PG96G-1.5-5-F—no PDL); Matek 96-wellglass bottom (Cat #PG96G-1.5-5-F—with PDL).

Media loss at 40 DIV: To measure how much media loss occurred at 40 DIVfor each plate, each plate was weighed at DIV 0 and then at DIV 40. Mostplates showed a similar media loss and none of them were significantlydifferent from the control plates.

Overall Performance: Plates that showed similar survival to the controlplates and had 96% or more focused images per plate were Nunc OpticalBottom 96-well (Cat #165305); Midsci 96-well (Cat #TP92096); and Matek96-well glass bottom (Cat #PG96G-1.5-5-F—no PDL).

The Midsci 96-well plate was selected for further testing because itgave focused images. 10 Midsci 96-well plates were imaged at DIV 4, 5,6, 7, 8, 9, 10, 11, 12, 13, 14. The number of focused images per platewas counted, and each had 95-96 wells in sharp focus for each DIV.

Example 3 Neuron Tracking

The following experiment was conducted to select parameters forsegmentation algorithms which would allow for automated neuron trackingover time. Several individuals were trained to visually identify neuronsin an image of a single well of a multi-well plate. These individualsthen selected neurons in each slice of a stack of the morphology channel(i.e., in multiple images of the same well taken over time). The neuronswere selected by selecting a point which signified roughly the brightestpoint of the neuron. Various segmentation algorithms were then comparedby determining and comparing the number of neurons detected by thealgorithm relative to the number of neurons selected by the humananalyzers. This provided an accuracy, false-positive, and false-negativerate for the segmentation algorithms. Parameters for the segmentationalgorithms were selected and/or adjusted until a false-negative of lessthan 10% was achieved. For example, when using a peak detectionalgorithm to identify peaks in the image histogram, a user-definedthreshold of 95% and or a peak region 2 fold greater than surroundingregions was selected. Peaks were defined as an absolute peak andsurrounding pixels within a defined range (e.g., all 8 adjacent pixels).Parameters were also adjusted to remove thin objects having a radius ofabout 10 pixels or less. In addition, a morphology filter having acircularity cutoff of 0.8 or higher was applied to eliminate objectssmaller than about 800 to 1000 pixels.

Example 4 Survival Times

The following experiment was conducted to select parameters foralgorithms which would allow for automated determination of neuronalsurvival times. Several individuals were trained to visually identifyneurons in an image of a single well of a multi-well plate. Theseindividuals then counted the survival times for neurons using themorphology channel for an experiment with a positive and negativecontrol (i.e. a situation where you would expect to see a survivaldifference). Hazard curves using survival statistics were generated fromthese hand-counted experiments. The hazard curves generated usingautomated segmentation were then compared to the hazard curves fromhand-counted experiments to determine how well the algorithms wereperforming. These trials were used to validate the eccentricity and areafilters, and suitable thresholds for such filters, in the automatedidentification of neurons from an image.

Example 5 Monitoring Mitochondrial Degradation Over Time in Neurons

Primary cortical neurons were transfected with two plasmids, EGFP and anew mitophagy reporter construct MitoEOS2, and cultured in a 96 wellplate. The plate was imaged at multiple time points using a roboticmicroscopy system as described herein. The fluorescence of the MitoEOS2construct was irreversibly shifted from green to red by exposure to apulse of blue light prior to imaging. The plate was then imaged eleventimes with the first seven images taken every four hours and the lastfour images separated by twenty-four hours. This is an additionalbenefit of the disclosed systems in that they allow for adjustment ofimaging parameters to capture biological processes that occur overdifferent timescales such as hours or days.

FIG. 23 provides images of one of several primary cortical neurons thatwere transfected and imaged as described above. The FITC (green) channel(top row) shows the morphology of the neuron which can be used as a maskfor determining signal intensity but can also be used for additionalimage analysis routines such as analysis of neurites as a readout ofneuron health. The RFP images (bottom row) show the same neuron shown inin the top row of images red-shifted by exposure to a pulse of bluelight at the beginning of imaging. The same neuron was imaged eleventimes with the first seven images taken every four hours and the lastfour images separated by twenty four hours. The top and bottom rows areimages of the same neuron at T1, T2, T3, T4, T5, and T6, wherein T2 is20 hr after T1, T3 is 24 hours after T1, T4 is 48 hr after T1, T5 is 72hr after T1, and T6 is 96 hours after T1.

As can be seen from the RFP images (bottom row), the signal from themitophagy reporter construct MitoEOS2 visibly decreases over timeindicating mitochondrial degradation. These images demonstrate theability of the systems disclosed herein to monitor changes in individualcells over time, e.g., during the normal life cycle of the cells or whenexposed to one or more experimental conditions. Furthermore, as can beseen from the images in FIG. 23, the disclosed systems allow monitoringand analysis of changes in cell morphology over time such as analysis ofneurites as a readout of neuron health.

Example 6 Automated Lipofectamine 2000 Transfection of Primary Neurons

A liquid handling workstation (MICROLAB® STARlet ML 8 96-prep system,available from Hamilton Robotics, Reno Nev.) was incorporated into theautomated microscope system described in Example 1, above. As aproof-of-concept, the liquid handling workstation was configured toperform Lipofectamine 2000 transfection of primary neurons as part ofthe automated system.

The reagents required were obtained from commercial sources and includedOpti-Mem, Lipofectamine 2000 Transfection Reagent, DNA, RNAi, NB\KY.Reagents were stored in reservoirs obtained from Seahorse Bioscience(Massachusetts, USA).

The system was configured to perform the following steps:

-   1. Pick up 96 (1-200 μl) tips.-   2. Aspirate 200 μl growth medium from Cell plate and dispense to    Cultured medium reservoir-   3. Pipette 200 μl per well (total 20 ml) from NB/KY reservoir and    dispense to Cell plate (dispensed on side as to not disturb cell    monolayer)-   4. Tip change, pick up 96 (1-200 μl) tips.-   5. Pipette 50 μl per well (total 5 ml) from Opti-MEM reservoir and    dispense 25 μl to DNA dilution plate and 25 μl to Lipofectamine    dilution plate.-   6. Tip change, pick up one column (1-200 μl) tips.-   7. Pipette 36 μl from Lipofectamine reservoir and dispense 3 μl    sequentially to each column of Lipfectamine dilution plate.-   8. Tip change, pick up 96 (1-200 μl) tips-   10. Tip change, pick up 96 (1-200 μl) tips.-   11. Pipette 1 μl from RNAi reservoir and dispense to DNA dilution    plate.-   12. Wait 5 min.-   13. Pipette 25 μl from DNA dilution plate and dispense drop-wise to    Lipofectamine dilution plate.-   14. Incubate at least 20 minutes at room temperature.-   15. Pipette 50 μl from Lipofectamine dilution plate and dispense    drop-wise to Cell plate.-   16. Transfer Cell plate off deck to incubator for incubation (20 m    to 3 hours).

Although the foregoing invention has been described in some detail byway of illustration and example for purposes of clarity ofunderstanding, it is readily apparent to those of ordinary skill in theart in light of the teachings of this invention that certain changes andmodifications may be made thereto without departing from the spirit orscope of the appended claims. It is also to be understood that theterminology used herein is for the purpose of describing particularembodiments only, and is not intended to be limiting, since the scope ofthe present invention will be limited only by the appended claims.

Accordingly, the preceding merely illustrates the principles of theinvention. It will be appreciated that those skilled in the art will beable to devise various arrangements which, although not explicitlydescribed or shown herein, embody the principles of the invention andare included within its spirit and scope. Furthermore, all examples andconditional language recited herein are principally intended to aid thereader in understanding the principles of the invention and the conceptscontributed by the inventors to furthering the art, and are to beconstrued as being without limitation to such specifically recitedexamples and conditions. Moreover, all statements herein recitingprinciples, aspects, and embodiments of the invention as well asspecific examples thereof, are intended to encompass both structural andfunctional equivalents thereof. Additionally, it is intended that suchequivalents include both currently known equivalents and equivalentsdeveloped in the future, i.e., any elements developed that perform thesame function, regardless of structure. The scope of the presentinvention, therefore, is not intended to be limited to the exemplaryembodiments shown and described herein. Rather, the scope and spirit ofpresent invention is embodied by the appended claims.

What is claimed is:
 1. An imaging system, the system comprising: asample holder comprising at least two walls defining a cutout portion,an internal beveled edge, and an internal bottom lip portion; an imagingdevice; and a robotic arm configured to automatically retrieve a samplefrom a first surface and place the sample on the imaging device, whereinthe system is configured to: automatically identify a fiduciary mark onthe sample, move the sample so that the fiduciary mark is insubstantially the same position as in a reference image, and acquire animage of the sample.
 2. The system according to claim 1, wherein therobotic arm is configured to automatically retrieve the sample from theimaging device and place the sample on a second surface.
 3. The systemaccording to claim 1, wherein the sample comprises a plate.
 4. Thesystem according to claim 3, wherein the plate is a multi-well plate. 5.The system according to claim 4, wherein the multi-well plate comprisesabout 96 wells or more.
 6. The system according to claim 3, wherein theplate comprises plastic.
 7. The system according to claim 6, wherein theplate is black.
 8. The system according to claim 1, further comprising abulk sample storage subsystem.
 9. The system according to claim 8,wherein the first surface is contained within the bulk sample storagesubsystem.
 10. The system according to claim 8, wherein the bulk samplestorage subsystem is configured to store 5 or more samples.
 11. Thesystem according to claim 8, wherein the bulk sample storage subsystemis configured to store 20 or more samples.
 12. The system according toclaim 8, wherein the bulk sample storage subsystem comprises a heatingelement configured to maintain the bulk sample storage subsystem at aspecified temperature.
 13. The system according to claim 8, wherein thebulk sample storage subsystem comprises a cooling element configured tomaintain the bulk sample storage subsystem at a specified temperature.14. The system according to claim 8, wherein the bulk sample storagesubsystem comprises a robotic arm configured to transfer a sample fromthe bulk sample storage subsystem to the first surface.
 15. The systemaccording to claim 8, wherein the bulk sample storage subsystemcomprises a robotic arm configured to transfer a sample from the secondsurface to the bulk sample storage subsystem.
 16. The system accordingto claim 1, wherein the robotic arm comprises a plurality of grippersconfigured to engage the sample.
 17. The system according to claim 16,wherein the grippers exert a lateral pressure on the sample.
 18. Thesystem according to claim 16, wherein the grippers comprise adjustableelements for engaging samples of different sizes.
 19. The systemaccording to claim 18, wherein the adjustable elements are manuallyadjustable.
 20. The system according to claim 1, further comprising asample identification subsystem.
 21. The system according to claim 20,wherein the sample identification subsystem comprises a barcode readerconfigured to read a barcode on the plate.
 22. The system according toclaim 20 wherein the sample identification subsystem is in electroniccommunication with a processor configured to identify the sample. 23.The system according to claim 22, wherein the processor is configured totailor the image acquisition steps for the sample.
 24. The systemaccording to claim 1, wherein the imaging device comprises an invertedmicroscope body.
 25. The system according to claim 24, wherein thesample is imaged from below.
 26. The system according to claim 1,wherein the fiduciary mark is located on the bottom of the sample. 27.The system according to claim 1, wherein the sample holder is attachedto a microscope stage of the imaging device.
 28. The system according toclaim 1, wherein the sample holder is removable from the imaging device.29. The system according to claim 1, wherein the internal beveled edgecomprises an angle relative to a plane of the bottom lip portion of thesample holder that is from about 85 deg. to about 25 deg.
 30. The systemaccording to claim 29, wherein the angle is from about 70 deg. to about40 deg.
 31. The system according to claim 1, wherein the sample holderis sized and shaped to receive a sample having a 127.5 mm×85 mmfootprint.
 32. The system according to claim 1, wherein the sampleholder comprises a sample receiving area comprising at least one cornerand an actuator configured to bias the sample into the at least onecorner.
 33. The system according to claim 1, wherein the imaging devicecomprises a camera having an exposure of 30 ms or less.
 34. The systemaccording to claim 1, wherein the camera is an EMCCD camera.
 35. Thesystem according to claim 1, wherein the imaging device comprises aXenon light source.
 36. The system according to claim 1, whereinacquiring an image of the sample comprises deconvolving amulti-wavelength image into its component wavelengths.
 37. An imagingsystem, the system comprising: a sample holder comprising at least twowalls defining a cutout portion, an internal beveled edge, and aninternal bottom lip portion; an imaging device; and a robotic armconfigured to automatically retrieve a sample from a first surface andplace the sample on the imaging device, wherein the system is configuredto: move the sample to an approximate location on a stage of the imagingdevice, automatically identify a fiduciary mark on the sample, move thesample from the approximate location until the fiduciary mark is alignedwith a reference image, acquire a plurality of images of the sample, theplurality of images being a temporal series of images of the sample, andtrack an object's movement within the sample via the plurality ofimages.